Job Discovery with Embeddings & Opportunity Finder Agent

[verdzy-kin]

Title: [BACKEND] Job Discovery with Embeddings & Opportunity Finder Agent

Integrate SerpAPI for job scraping, implement semantic matching with embeddings, and create Opportunity Finder Agent.

[verdzy-kin]

Acceptance Criteria:

• SerpAPI integration for job scraping
• Parse job results into standardized format
• Text embeddings integration (text-embedding-3-small)
• Redis caching for embeddings (weekly refresh)
• Opportunity Finder Agent:
  • Takes user_id as input
  • Fetches user profile (skills, interests, location)
  • Scrapes jobs matching criteria
  • Calculates similarity score using embeddings
  • Ranks jobs by relevance (0-100)
  • Returns top 20 opportunities
• GET /jobs/discover returns recommendations
• Caching layer to reduce API calls
• Rate limiting for SerpAPI (free tier: 100/month)
• Daily digest email option
• Unit tests with mock data
• Cost tracking for embeddings
• Swagger docs updated