Anish Sarkar
d800ca33a5
feat(instagram): handle soft login wall in fetch logic
...
Added detection for Instagram's soft login wall, which returns a 200 status with login HTML. Implemented a new function to identify login redirects and adjusted the fetch logic to treat these cases similarly to 401/403 responses.
2026-07-09 18:29:17 +05:30
Anish Sarkar
ba83d16cba
docs(web): add Instagram to MCP server tool docs
2026-07-09 16:07:50 +05:30
Anish Sarkar
f5aad487ac
docs(web): document Instagram native connector
2026-07-09 16:07:42 +05:30
Anish Sarkar
4e86710b26
feat(web): cross-link Instagram from sibling connectors
2026-07-09 16:07:34 +05:30
Anish Sarkar
fa2ac406f3
feat(web): add Instagram connector marketing page
2026-07-09 16:07:19 +05:30
Anish Sarkar
d00030d97c
feat(web): add Instagram playground platform and brand icon
2026-07-09 16:03:34 +05:30
Anish Sarkar
7309ca5ca0
test(instagram): add manual e2e scraper script
2026-07-09 16:03:29 +05:30
Anish Sarkar
e151346fbe
test(chat): include Instagram in subagent composition test
2026-07-09 16:03:23 +05:30
Anish Sarkar
458d223ef8
test(billing): cover Instagram item and comment billing
2026-07-09 16:02:54 +05:30
Anish Sarkar
2e66e714e2
test(instagram): add capability unit tests
2026-07-09 16:02:48 +05:30
Anish Sarkar
9da136070d
test(instagram): add platform-layer unit tests
2026-07-09 16:02:41 +05:30
Anish Sarkar
98eab8c4b5
feat(chat): add Instagram builtin subagent
2026-07-09 16:02:05 +05:30
Anish Sarkar
c5e3fd4006
feat(mcp): add Instagram scrape/comments/details tools
2026-07-09 16:01:55 +05:30
Anish Sarkar
929efe8152
feat(instagram): register instagram.* capability namespace
2026-07-09 16:01:49 +05:30
Anish Sarkar
7ea474be93
feat(instagram): add details capability verb
2026-07-09 16:01:38 +05:30
Anish Sarkar
38a26b1ccf
feat(instagram): add comments capability verb
2026-07-09 14:58:58 +05:30
Anish Sarkar
ff55537bce
feat(instagram): add scrape capability verb
2026-07-09 14:58:54 +05:30
Anish Sarkar
eccc142974
docs(instagram): add platform scraper README
2026-07-09 14:58:47 +05:30
Anish Sarkar
6270250f74
feat(instagram): add fan-out orchestrator with deterministic early-stop cleanup
2026-07-09 14:58:38 +05:30
Anish Sarkar
b1eff478fd
feat(instagram): add proxy-aware fetch with warm-up and rotate-on-block
2026-07-09 14:58:32 +05:30
Anish Sarkar
2d29001004
feat(instagram): add web-JSON parsers
2026-07-09 14:58:27 +05:30
Anish Sarkar
8dc8ad6a36
feat(instagram): add URL classifier and normalizer
2026-07-09 14:58:15 +05:30
Anish Sarkar
842a0e666d
feat(instagram): add platform input/output schemas
2026-07-09 14:58:11 +05:30
Anish Sarkar
51c2dfdfe6
feat(config): add Instagram per-item and per-comment scrape rates
2026-07-09 14:57:59 +05:30
Anish Sarkar
43660ee51b
feat(billing): map Instagram units to rate keys and display nouns
2026-07-09 14:57:53 +05:30
Anish Sarkar
f204db7bce
feat(billing): add INSTAGRAM_ITEM and INSTAGRAM_COMMENT units
2026-07-09 14:57:44 +05:30
Anish Sarkar
8486a4f914
feat(constants): add Instagram to SUBAGENT_TO_REQUIRED_CONNECTOR_MAP for enhanced multi-agent chat capabilities
2026-07-09 14:08:23 +05:30
Anish Sarkar
3bd864ed95
feat(env): add Instagram scraping configuration parameters to .env.example
2026-07-09 14:08:10 +05:30
Rohan Verma
3eb485ff31
Merge pull request #1575 from SyntaxSawdust/feature/1354-embedding-base-url
...
Support separate embedding base URL configuration
2026-07-08 17:11:42 -07:00
Rohan Verma
d2ed0bb382
Merge pull request #1570 from pelazas/fix/scheduler-task-map-bookstack
...
fix: dispatch periodic BookStack indexing from the scheduler task_map
2026-07-08 15:07:38 -07:00
DESKTOP-RTLN3BA\$punk
b8155c6c0e
Merge commit ' 1ae927a7db' into dev
2026-07-08 15:05:11 -07:00
CREDO23
ba375ea3ee
feat(tiktok): emit graceful ErrorItem for blocked/empty listings
...
Profile and search feeds are trust-gated: an anonymous headless session
gets an empty item_list (profile) or no results XHR (search), while
hashtag feeds load. A zero-item listing now yields one honest ErrorItem
(errorCode="no_items") instead of vanishing silently, and ErrorItems are
excluded from billing so a blocked target is surfaced but never charged.
2026-07-08 23:14:50 +02:00
CREDO23
f731d371ab
feat(tiktok): add TikTok SEO landing page and site-wide marketing parity
2026-07-08 20:36:46 +02:00
CREDO23
5f9472b122
feat(tiktok): advertise tiktok across agent prompts, MCP instructions, docs
2026-07-08 20:30:38 +02:00
CREDO23
ea6befdf59
docs(tiktok): native connector page + nav card
2026-07-08 20:21:29 +02:00
CREDO23
ed1c3a1f3d
feat(tiktok): expose tiktok.scrape on MCP, chat subagent, playground
2026-07-08 20:20:20 +02:00
CREDO23
2943d8b23c
feat(tiktok): tiktok.scrape capability + billing wire-up
2026-07-08 18:21:44 +02:00
CREDO23
9dd39faa50
test(tiktok): pin live fixtures, trim listing debug scaffolding
2026-07-08 17:53:29 +02:00
CREDO23
ea95a0529f
test(tiktok): live e2e smoke script
2026-07-08 17:14:13 +02:00
CREDO23
5a326f3818
fix(tiktok): warm browser listing + drop ctx-var-in-generator
2026-07-08 17:14:13 +02:00
CREDO23
ad8b73477a
feat(tiktok): browser-driven signed listings
2026-07-08 16:37:36 +02:00
CREDO23
daa0856c44
feat(tiktok): blob-first orchestrator + video flow
2026-07-08 16:07:07 +02:00
CREDO23
44b3e640d3
feat(tiktok): rotate-on-block fetch session
2026-07-08 15:51:17 +02:00
CREDO23
5688ab0678
feat(tiktok): scrape input schema
2026-07-08 15:51:17 +02:00
CREDO23
ef66063bfc
feat(tiktok): video normalizer and parser tests
2026-07-08 15:40:50 +02:00
CREDO23
0bbcf99852
feat(tiktok): author normalizer
2026-07-08 15:40:38 +02:00
CREDO23
8840fd2c53
feat(tiktok): rehydration blob extractor
2026-07-08 15:40:21 +02:00
CREDO23
b2d387f994
feat(tiktok): URL target resolver
2026-07-08 15:40:07 +02:00
CREDO23
ef3f9f2e25
feat(tiktok): item output schemas
2026-07-08 15:39:58 +02:00
CREDO23
3c027b9f49
Merge remote-tracking branch 'upstream/main' into dev
...
# Conflicts:
# surfsense_mcp/mcp_server/core/auth/__init__.py
# surfsense_mcp/mcp_server/core/auth/headers.py
# surfsense_mcp/mcp_server/core/auth/identity.py
# surfsense_mcp/mcp_server/core/auth/middleware.py
# surfsense_mcp/mcp_server/core/transport/__init__.py
# surfsense_mcp/mcp_server/core/transport/http.py
# surfsense_mcp/mcp_server/core/workspace_matching.py
# surfsense_mcp/mcp_server/features/knowledge_base/annotations.py
# surfsense_mcp/mcp_server/features/knowledge_base/document_tools.py
# surfsense_mcp/mcp_server/features/knowledge_base/search_tools.py
# surfsense_mcp/mcp_server/features/scrapers/annotations.py
# surfsense_mcp/mcp_server/features/scrapers/platforms/__init__.py
# surfsense_mcp/mcp_server/features/scrapers/platforms/google_maps.py
# surfsense_mcp/mcp_server/features/scrapers/platforms/google_search.py
# surfsense_mcp/mcp_server/features/scrapers/platforms/reddit.py
# surfsense_mcp/mcp_server/features/scrapers/platforms/web.py
# surfsense_mcp/mcp_server/features/scrapers/platforms/youtube.py
# surfsense_mcp/mcp_server/features/scrapers/run_history.py
# surfsense_mcp/src/surfsense_mcp/__main__.py
# surfsense_mcp/src/surfsense_mcp/features/knowledge_base/__init__.py
# surfsense_mcp/src/surfsense_mcp/features/scrapers/__init__.py
# surfsense_mcp/tests/test_auth_headers.py
# surfsense_mcp/tests/test_request_auth.py
# surfsense_mcp/tests/test_workspace_context.py
# surfsense_web/content/docs/how-to/mcp-server.mdx
# surfsense_web/lib/mcp/clients.ts
2026-07-08 14:52:55 +02:00
