SurfSense/surfsense_backend/app/tasks
DESKTOP-RTLN3BA\$punk 17bdb06825 feat(03c): bill credits per successful web crawl
Add a config-driven crawl meter (1 USD / 1000 successes by default, retunable
via WEB_CRAWL_MICROS_PER_SUCCESS) mirroring EtlCreditService:

- WebCrawlCreditService (gate -> pre-check -> post-charge) on the unified wallet
- webcrawler indexer: pre-flight credit block + post-charge of the workspace
  owner per success, with a web_crawl TokenUsage audit row
- chat scrape_webpage (main + research): fold each successful scrape into the
  active turn's token accumulator so it settles with the chat turn
- WEB_CRAWL_CREDIT_BILLING_ENABLED / WEB_CRAWL_MICROS_PER_SUCCESS config + env
- unit tests (service) + integration tests (indexer wiring)
- scripts/e2e_phase3_crawl_billing.py: manual functional e2e (3a/3b/3c)

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-29 21:40:53 -07:00
..
celery_tasks refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
chat feat(03a): implement Scrapling-only web crawler and remove Firecrawl integration 2026-06-29 20:37:56 -07:00
connector_indexers feat(03c): bill credits per successful web crawl 2026-06-29 21:40:53 -07:00
document_processors Merge commit '683a7d3bc2' into feat/rename-searchspace-to-workspace 2026-06-29 17:55:15 -07:00
__init__.py feat: SurfSense v0.0.6 init 2025-03-14 18:53:14 -07:00
composio_indexer.py refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00