🌐 APIs & Web Scraping

Description: Make GET requests to JSONPlaceholder, a free fake API for testing.

• Use requests.get() to fetch data from https://jsonplaceholder.typicode.com/posts
• Print the first 3 posts with their titles and body text
• Fetch a single user from /users/1 and display their name, email, and company
• Use raise_for_status() to handle potential errors
• Add a timeout of 5 seconds to your requests

Bonus: Use query parameters to filter posts by userId: params={'userId': 1}

Description: Parse nested JSON responses from a weather-like API structure.

• Create a sample nested JSON structure simulating weather data with location, current, and forecast keys
• Extract the current temperature and weather condition from the nested structure
• Loop through a 5-day forecast array and print each day's high and low temperatures
• Handle missing keys gracefully using .get() with default values
• Convert the parsed data into a Pandas DataFrame

Bonus: Use json.dumps(data, indent=2) to pretty-print JSON for debugging.

Description: Work with authenticated API requests using the GitHub API.

• Create a personal access token on GitHub (Settings → Developer Settings → Tokens)
• Store the token in an environment variable (os.getenv('GITHUB_TOKEN'))
• Make an authenticated request to https://api.github.com/user using the Authorization header
• Fetch your public repositories and display their names and star counts
• Handle 401 (unauthorized) and 403 (rate limit) errors appropriately

Bonus: Check remaining rate limits using response.headers['X-RateLimit-Remaining'].

Description: Build a BeautifulSoup scraper to extract headlines from a practice news site.

• Target https://quotes.toscrape.com (a safe practice scraping site)
• Extract all quotes, their authors, and associated tags from the homepage
• Use find_all() to locate all quote containers
• Handle missing tags gracefully (check for None before accessing .text)
• Store results in a list of dictionaries with keys: quote, author, tags
• Export the scraped data to a CSV file using Pandas

Advanced Challenge:

• Implement pagination by finding and following the "Next" button link
• Scrape all 10 pages of quotes
• Add time.sleep(1) between page requests to be polite
• Create a function scrape_page(url) that returns a list of quotes from a single page

Description: Build a production-ready API client that handles errors, timeouts, and rate limits gracefully.

• Create a function fetch_with_retry(url, max_retries=3) that retries failed requests
• Implement exponential backoff: wait 1s, 2s, 4s between retries
• Handle specific HTTP status codes:
  • 429 Too Many Requests: Wait and retry based on Retry-After header
  • 500-503 Server Errors: Retry with backoff
  • 401/403 Auth Errors: Log error and don't retry
• Add timeout handling with timeout=10 seconds
• Log all attempts and failures for debugging

Bonus: Use the tenacity library for advanced retry logic with decorators.

Description: Create a complete data pipeline that respects API rate limits.

• Build a pipeline that fetches 100 posts from JSONPlaceholder API
• Implement a rate limiter: maximum 10 requests per minute
• Use requests.Session() for connection pooling
• Add logging to track request timing and any errors
• Save progress to a checkpoint file so you can resume if interrupted
• Export final results to both JSON and CSV formats

Advanced: Use asyncio with aiohttp for concurrent requests while still respecting rate limits.