CREDO23
3312a442f3
refactor(proprietary): move scrapers/ -> platforms/ (one subpackage per platform)
2026-07-03 11:01:58 +02:00
CREDO23
9fe9c5b71d
feat(capabilities): replace web.scrape/web.discover with unified web.crawl
...
web.crawl scrapes a single URL (maxCrawlDepth=0) or spiders a whole site,
backed by the proprietary site_crawler engine. Rewires the scraping subagent
tools and capability tests onto the new verb.
2026-07-03 10:51:05 +02:00
CREDO23
f82fae3973
feat(web-crawler): add site_crawler spider + url_policy over the fetch engine
2026-07-03 10:50:41 +02:00
CREDO23
5d0bd8125b
test(capabilities): cover youtube verb schemas, executors, and registry
2026-07-02 20:18:53 +02:00
CREDO23
f967cebe92
docs(scraping-agent): surface youtube in description and prompt
2026-07-02 20:18:53 +02:00
CREDO23
c467a99dc6
feat(scraping-agent): wire youtube verbs into subagent tools
2026-07-02 20:18:53 +02:00
CREDO23
0b2dba7a57
feat(capabilities): expose youtube.* verbs on the REST door
2026-07-02 20:18:53 +02:00
CREDO23
6c3735db19
feat(capabilities): add youtube.scrape and youtube.comments verbs
2026-07-02 20:18:53 +02:00
CREDO23
af3e70ea56
Merge upstream/ci_mvp: add YouTube scraper actor under app/proprietary
...
Resolved modify/delete conflicts on the old 'revamp phases 4-7/' plan docs
by keeping our deletion (superseded by flattened plans/backend/00-*.md).
2026-07-02 20:01:11 +02:00
CREDO23
6bd0d640e6
refactor(capabilities): group framework into core/, keep verbs top-level
2026-07-02 19:54:20 +02:00
CREDO23
87df0bb629
fix(web.discover): cap results to top_k
2026-07-02 19:40:37 +02:00
CREDO23
9738392161
refactor(agents): rename intelligence_agent subagent to scraping
2026-07-02 18:59:34 +02:00
CREDO23
790507d107
feat(capabilities): meter captcha, truncate scrape, bound discover
2026-07-02 18:32:42 +02:00
CREDO23
18e2ad13ca
test(automations): cover checkpointer cross-loop durability
2026-07-02 17:54:31 +02:00
CREDO23
f0c8d9ba5c
test(chat): cover watch tools
2026-07-02 17:54:31 +02:00
CREDO23
f1ea60365c
test(automations): cover watch rest routes
2026-07-02 17:54:31 +02:00
CREDO23
555dc3fcad
test(automations): cover chat watch service
2026-07-02 17:54:31 +02:00
CREDO23
61412163fd
test(automations): cover chat_message action
2026-07-02 17:54:31 +02:00
CREDO23
fecb4fa05e
test(automations): add integration harness
2026-07-02 17:54:31 +02:00
CREDO23
3278f46585
fix(tasks): dispose checkpointer pool per celery task
2026-07-02 17:54:31 +02:00
CREDO23
896f38fba1
feat(chat): wire watch tools into intelligence agent
2026-07-02 17:54:31 +02:00
CREDO23
32d542f7a3
feat(chat): add stop and refresh watch tools
2026-07-02 17:54:31 +02:00
CREDO23
cf546b6821
feat(chat): add start_watch tool
2026-07-02 17:54:31 +02:00
CREDO23
c0c0b13029
feat(automations): add watch rest routes
2026-07-02 17:54:31 +02:00
CREDO23
2c9862c315
feat(automations): add chat watch service
2026-07-02 17:54:31 +02:00
CREDO23
b993bf31d9
feat(automations): add chat_message action
2026-07-02 17:54:31 +02:00
DESKTOP-RTLN3BA\$punk
cc99bc4cae
Update licensing information for SurfSense software
...
- Added Business Source License 1.1 details to the proprietary LICENSE file.
- Clarified terms of use, including production use restrictions and transition to Apache License after four years.
- Updated README.md to reflect the new licensing structure and its implications for users.
This change ensures clear communication of licensing terms for proprietary components within the SurfSense project.
2026-07-02 03:28:42 -07:00
DESKTOP-RTLN3BA\$punk
0445c646c6
refactor(native-connector): relocate youtube scraper under app/proprietary + parallelize playlists
...
Move app/scrapers -> app/proprietary/scrapers/youtube to sit alongside the existing proprietary web_crawler/platforms namespace, updating all external imports (routes, tests, e2e script, README). Internal imports were relative so are unchanged.
Also parallelize playlist per-video resolution: page video ids sequentially, then resolve the heavy watch-page fetches concurrently via fan_out (~150 videos ~70s, down from a few minutes). Items stream in completion order; sort by the order field for playlist order.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-02 02:34:04 -07:00
DESKTOP-RTLN3BA\$punk
99378d7b63
feat(native-connector): added youtube scrapers
2026-07-02 01:10:31 -07:00
CREDO23
7ad83760b5
feat(chat): add intelligence_agent subagent over capability verbs
2026-07-02 00:17:22 +02:00
CREDO23
110beb4dd1
feat(chat): add web_scrape/web_discover tool emission handlers
2026-07-02 00:17:10 +02:00
CREDO23
de4b8ab6b0
feat(capabilities): generate agent tool door from registry
2026-07-02 00:17:01 +02:00
CREDO23
23802a74bb
feat(capabilities): document verbs with descriptions and field docs
2026-07-02 00:16:55 +02:00
CREDO23
b1bd35c082
test(capabilities): cover REST door, meter-gate, and batch bounds
2026-07-01 17:53:42 +02:00
CREDO23
b1e49f21f2
feat(capabilities): mount capability REST routes
2026-07-01 17:53:42 +02:00
CREDO23
9c9ad97f8e
feat(capabilities): generate REST door with per-workspace rate limit
2026-07-01 17:53:42 +02:00
CREDO23
c696829568
feat(capabilities): bound web.scrape batch and expose worst-case units
2026-07-01 17:53:42 +02:00
CREDO23
8fbfa5a6e1
feat(capabilities): add pre-flight meter-gate for billed verbs
2026-07-01 17:53:42 +02:00
CREDO23
f257e385db
feat(capabilities): register web.discover verb
2026-07-01 17:08:58 +02:00
CREDO23
329bc63b6c
feat(capabilities): add env-keyed searxng/linkup/baidu discover providers
2026-07-01 17:08:58 +02:00
CREDO23
adc39d1062
feat(config): add env keys for discover search providers
2026-07-01 17:08:58 +02:00
CREDO23
fa1055dd4c
feat(capabilities): add web.discover schemas and provider seam
2026-07-01 17:08:58 +02:00
CREDO23
a413539f6a
feat(capabilities): allow free verbs with no billing unit
2026-07-01 17:08:58 +02:00
CREDO23
8d76afc9d6
test(capabilities): cover registry, executor, schemas, billing
2026-07-01 16:42:47 +02:00
CREDO23
6b91a7ed0c
feat(capabilities): register web.scrape verb
2026-07-01 16:42:47 +02:00
CREDO23
6679fda662
feat(capabilities): wire web.scrape to proprietary crawler
2026-07-01 16:42:47 +02:00
CREDO23
bbd81bcc99
feat(capabilities): add web.scrape schemas
2026-07-01 16:42:47 +02:00
CREDO23
bfe3117302
feat(capabilities): bill workspace owner at executor
2026-07-01 16:42:47 +02:00
CREDO23
3e6007431d
feat(capabilities): add in-process verb store
2026-07-01 16:42:47 +02:00
CREDO23
78f7a93a88
feat(capabilities): add registry contracts and package
2026-07-01 16:42:47 +02:00
