apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-06 22:12:12 +02:00
Code Issues Projects Releases Packages Wiki Activity
SurfSense/surfsense_backend
History
DESKTOP-RTLN3BA\$punk 17bdb06825 feat(03c): bill credits per successful web crawl 
Add a config-driven crawl meter (1 USD / 1000 successes by default, retunable
via WEB_CRAWL_MICROS_PER_SUCCESS) mirroring EtlCreditService:

- WebCrawlCreditService (gate -> pre-check -> post-charge) on the unified wallet
- webcrawler indexer: pre-flight credit block + post-charge of the workspace
  owner per success, with a web_crawl TokenUsage audit row
- chat scrape_webpage (main + research): fold each successful scrape into the
  active turn's token accumulator so it settles with the chat turn
- WEB_CRAWL_CREDIT_BILLING_ENABLED / WEB_CRAWL_MICROS_PER_SUCCESS config + env
- unit tests (service) + integration tests (indexer wiring)
- scripts/e2e_phase3_crawl_billing.py: manual functional e2e (3a/3b/3c)

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-29 21:40:53 -07:00
..
alembic Merge commit '683a7d3bc2' into feat/rename-searchspace-to-workspace 2026-06-29 17:55:15 -07:00
app feat(03c): bill credits per successful web crawl 2026-06-29 21:40:53 -07:00
scripts feat(03c): bill credits per successful web crawl 2026-06-29 21:40:53 -07:00
tests feat(03c): bill credits per successful web crawl 2026-06-29 21:40:53 -07:00
.dockerignore chore(backend): exclude tests/ from production Docker image 2026-05-06 17:16:22 +05:30
.env.example feat(03c): bill credits per successful web crawl 2026-06-29 21:40:53 -07:00
.gitignore fix(gitignore): anchor data/ rule; track podcast voice catalogs 2026-06-12 00:06:37 +02:00
.python-version feat: SurfSense v0.0.6 init 2025-03-14 18:53:14 -07:00
alembic.ini add github connector, add alembic for db migrations, fix bug updating connectors 2025-04-13 13:56:22 -07:00
celery_worker.py fix: celery_app path and gmail indexing 2025-10-21 21:11:41 -07:00
Dockerfile feat(03a): implement Scrapling-only web crawler and remove Firecrawl integration 2026-06-29 20:37:56 -07:00
main.py feat(observability): add OpenTelemetry process bootstrap 2026-05-21 23:01:54 +05:30
pyproject.toml feat(03a): implement Scrapling-only web crawler and remove Firecrawl integration 2026-06-29 20:37:56 -07:00
uv.lock feat(03a): implement Scrapling-only web crawler and remove Firecrawl integration 2026-06-29 20:37:56 -07:00