apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-06 22:12:12 +02:00
Code Issues Projects Releases Packages Wiki Activity
SurfSense/surfsense_backend/app
History
DESKTOP-RTLN3BA\$punk 17bdb06825 feat(03c): bill credits per successful web crawl 
Add a config-driven crawl meter (1 USD / 1000 successes by default, retunable
via WEB_CRAWL_MICROS_PER_SUCCESS) mirroring EtlCreditService:

- WebCrawlCreditService (gate -> pre-check -> post-charge) on the unified wallet
- webcrawler indexer: pre-flight credit block + post-charge of the workspace
  owner per success, with a web_crawl TokenUsage audit row
- chat scrape_webpage (main + research): fold each successful scrape into the
  active turn's token accumulator so it settles with the chat turn
- WEB_CRAWL_CREDIT_BILLING_ENABLED / WEB_CRAWL_MICROS_PER_SUCCESS config + env
- unit tests (service) + integration tests (indexer wiring)
- scripts/e2e_phase3_crawl_billing.py: manual functional e2e (3a/3b/3c)

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-29 21:40:53 -07:00
..
agents feat(03c): bill credits per successful web crawl 2026-06-29 21:40:53 -07:00
auth chore: fix linting 2026-06-25 04:31:22 +05:30
automations feat(03a): implement Scrapling-only web crawler and remove Firecrawl integration 2026-06-29 20:37:56 -07:00
config feat(03c): bill credits per successful web crawl 2026-06-29 21:40:53 -07:00
connectors feat(03a): implement Scrapling-only web crawler and remove Firecrawl integration 2026-06-29 20:37:56 -07:00
etl_pipeline chore: linting 2026-06-17 22:31:36 -07:00
event_bus refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
file_storage refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
gateway refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
indexing_pipeline refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
notifications refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
observability refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
podcasts refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
prompts prompts: remove orphaned system_prompt_composer surface 2026-06-25 18:37:04 +02:00
proprietary feat(03b): add BYO custom proxy provider + bounded crawler rotation-retry 2026-06-29 21:03:34 -07:00
retriever refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
routes refactor(routes): rename to workspace + consolidate URL spellings (Phase 2 Wave E) 2026-06-26 18:35:08 +02:00
schemas refactor(schemas): rename SearchSpace -> Workspace in Pydantic schemas (Phase 2 Wave C) 2026-06-26 18:23:43 +02:00
services feat(03c): bill credits per successful web crawl 2026-06-29 21:40:53 -07:00
tasks feat(03c): bill credits per successful web crawl 2026-06-29 21:40:53 -07:00
templates feat: update report generation and export capabilities to support multiple formats (PDF, DOCX, HTML, LaTeX, EPUB, ODT, plain text) across documentation and backend 2026-03-09 18:41:21 -07:00
utils feat(03b): add BYO custom proxy provider + bounded crawler rotation-retry 2026-06-29 21:03:34 -07:00
__init__.py feat: SurfSense v0.0.6 init 2025-03-14 18:53:14 -07:00
app.py refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
celery_app.py fix(auth):schedule refresh token cleanup 2026-06-23 12:50:02 +05:30
db.py refactor(db): rename SearchSpace ORM core to Workspace (Phase 2 Wave A) 2026-06-26 18:18:50 +02:00
exceptions.py feat: add processing mode support for document uploads and ETL pipeline, improded error handling ux 2026-04-14 21:26:00 -07:00
rate_limiter.py try: ip fix for cludflare 2026-04-16 02:13:52 -07:00
session_events.py refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
users.py refactor(backend): rename search_space -> workspace across app bulk (Phase 2 Wave D) 2026-06-26 18:30:47 +02:00
zero_publication.py refactor(db): publish workspace_id column in zero_publication 2026-06-26 16:56:39 +02:00