Resolves: surfsense_backend/app/agents/new_chat/middleware/memory_injection.py
- Took both imports: upstream moved MEMORY_HARD_LIMIT/SOFT_LIMIT to
app.services.memory; kept our perf-logger import for timing.
Pulls in upstream changes:
- Memory document feature (services/memory refactor, removal of
app.agents.new_chat.memory_extraction and background extraction in
stream_new_chat — agent now drives memory via update_memory tool).
- BACKEND_URL env refactor across web tool-ui/editor/chat/dashboard/lib.
- GitHub Actions backend test workflow + pre-commit biome bump.
- Token-display polish in MessageInfoDropdown; save_memory no-update
sentinel.
Verified: 1723 unit tests pass, ruff clean. No semantic regression in
stream_new_chat (their memory-extraction deletion and our preflight
removal touch different functions).
Collapse the invalidate + warmup pair into a single
refresh_mcp_tools_cache_for_connector(connector_id, search_space_id)
helper and scope live discovery to the one connector that changed
instead of the whole search space.
- new mcp_tool.discover_single_mcp_connector: load one connector,
refresh OAuth if needed, force live MCP discovery so its cached_tools
row is rewritten; returned wrappers are discarded since the in-process
LRU is rebuilt lazily on the next user query
- mcp_tools_cache.refresh_mcp_tools_cache_for_connector: synchronously
evicts the per-space LRU (LRU keys cannot scope finer) and schedules
the per-connector prefetch via loop.create_task
- routes (OAuth callback, MCP POST, MCP PUT) collapse their two
back-to-back calls into a single refresh call; DELETE handlers keep
using bare invalidate_mcp_tools_cache (nothing to prefetch)
No new automated tests: the new functions are I/O glue (DB + network)
where mocked unit tests would test implementation rather than behavior.
The existing 9 unit tests for the cached_tools data shape are unchanged.
- Updated the `_get_metadata` function to handle changes in the Stripe SDK, specifically for `StripeObject` which is no longer a subclass of `dict` in `stripe>=15.0`.
- Implemented a fallback mechanism in `finalize_checkout` to recover purchase type from the database if metadata extraction fails, ensuring robust handling of checkout sessions.
- Added a new endpoint `/stripe/finalize-checkout` to synchronously fulfill a checkout session, addressing the webhook-vs-redirect race condition.
- Updated the `PurchaseSuccessPage` component to handle various states of the checkout process, including loading, completed, pending, and failed states.
- Introduced a new response model `FinalizeCheckoutResponse` to provide immediate feedback on the purchase status.
- Enhanced the Stripe API service to include the new finalize checkout functionality.
Closes#1295
The connector indexing route's `_run_indexing_with_notifications` set the
Redis heartbeat key once at the start of indexing and relied on
`on_heartbeat_callback` (only fired in Phase 2 per-document loops) to
refresh it. The GitHub connector's Phase 1 runs `gitingest` as a blocking
subprocess via `asyncio.to_thread`, so for any repo larger than the
2-minute TTL, the key expires before Phase 2 starts. The
`cleanup_stale_indexing_notifications_task` then marks the document as
failed with the misleading "Sync was interrupted unexpectedly. Please
retry." message — even though the indexing thread is still running and
gitingest's own subprocess timeout is 900 seconds.
Add a background asyncio coroutine that refreshes the Redis key every
60 seconds for the duration of the indexing call. Same pattern already
in use at app/tasks/celery_tasks/document_tasks.py:_run_heartbeat_loop,
just adapted to use the route's get_heartbeat_redis_client() and
_get_heartbeat_key() helpers.
Cancellation runs in the `finally` block BEFORE the heartbeat-key
delete so the loop cannot race and re-create the key after we have
deleted it. The new `HEARTBEAT_REFRESH_INTERVAL = 60` constant mirrors
the celery task module's value.
- Included server_time_utc in the connect response schema for better synchronization.
- Updated obsidian_connect function to set server_time_utc during connection handling.
- Enhanced integration tests to verify the presence of server_time_utc in responses.
- Improved connectivity status recovery in the sync engine for better error management.
