- Add anthropic_model and openai_model columns to database
- Allow users to select their preferred AI model in settings
- Update defaults to current models (Claude Haiku 4.5, GPT-4.1 Nano)
- Include model options: Claude 4.5 series, GPT-4.1/5.1 series
- Pass user-selected model to all AI extraction/verification functions
- Log which model is being used for debugging
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sonnet 4 is ~4-12x more expensive than Haiku 3.5.
Haiku 3.5 should still be capable enough for price extraction.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Haiku was being too literal in price extraction, often "correcting"
valid alternative prices (like Amazon other sellers) to the main
buy box price.
Sonnet is more capable and should handle nuanced cases better.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix TypeScript narrowing issue in Best Buy scraper
- Add browser rendering for JS-heavy sites (Best Buy, Target, Walmart, Costco)
- Improve Best Buy site-specific scraper with better selectors and logging
- Skip payment plan prices (/mo, per month, etc.)
- Enhance AI HTML preparation:
- Extract JSON-LD data before removing scripts
- Add price element extraction for better context
- Increase character limit from 15000 to 25000
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add multi-strategy voting: runs JSON-LD, site-specific, generic CSS,
and AI extraction methods in parallel
- Implement consensus voting to select the correct price when methods agree
- Add AI arbitration when extraction methods disagree
- Add PriceSelectionModal for users to select correct price when ambiguous
- Store preferred extraction method per product for faster re-checks
- Add database columns for preferred_extraction_method and needs_price_review
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- AI verification now checks if product is purchasable RIGHT NOW
- AI returns stockStatus field for pre-order/coming soon detection
- Enhanced generic stock status detection with pre-order phrases:
- Coming soon, available soon, releases on, pre-order, etc.
- Notify me when available, join waitlist, etc.
- Pre-order buttons no longer count as Add to Cart
- Fixes issue where unreleased products showed as in stock
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When enabled, AI verifies every scraped price to ensure accuracy.
This catches issues like scraped "savings" amounts instead of actual prices.
- Add ai_verification_enabled column to users table
- Create verification prompt and functions for Anthropic, OpenAI, Ollama
- Integrate verification step into scraper after traditional scraping
- Add verification toggle to Settings page (separate from AI extraction)
- AI verification is independent of AI extraction fallback
Flow: Traditional scraping -> AI verification (if enabled) -> AI extraction fallback (if no price found)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add database migration for ollama_base_url and ollama_model columns
- Update backend models and queries for Ollama settings
- Add extractWithOllama function using Ollama's /api/chat endpoint
- Add /api/settings/ai/test-ollama endpoint to test connection and list models
- Update frontend Settings page with Ollama configuration UI
- Support model selection from dropdown after testing connection
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Extract JSON-LD scripts BEFORE removing script tags
- Add logging for prepared HTML, AI responses, and parsed data
- Include more detailed error messages in test endpoint
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add AI extraction service supporting Anthropic (Claude) and OpenAI
- Add AI settings UI in Settings page with provider selection
- Add database migration for AI settings columns
- Integrate AI fallback into scraper when standard methods fail
- Add API endpoints for AI settings and test extraction
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
