Skip to content

Implement Polite fetch_page() #24

@ThomasKarpinski

Description

@ThomasKarpinski

Implement the core function for making HTTP requests to retrieve HTML content safely. Tasks:

  • Implement fetch_page(url) using requests.
  • Include custom User-Agent header (polite scraping).
  • Ensure error handling for connection issues and status codes (e.g., resp.raise_for_status()).

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions