mirror of
https://github.com/VectifyAI/PageIndex.git
synced 2026-07-15 21:11:05 +02:00
890b520b1c
Real concurrency e2e (8 threads adding the same file) surfaced a bug the mocked unit tests missed: concurrent add_document calls failed with sqlite3.OperationalError "database is locked". Root cause — under WAL the dedup SELECT (find_document_by_hash) left a read snapshot on the connection, and the subsequent INSERT on that stale snapshot raised SQLITE_BUSY_SNAPSHOT, which busy_timeout does not retry. Fixes: - open connections in autocommit (isolation_level=None) so a SELECT never leaves a lingering read snapshot and each write is its own transaction - PRAGMA busy_timeout=10000 so concurrent writers wait for the WAL single-writer lock instead of failing immediately - an instance-level write lock serializing the fast write methods within the process (the expensive LLM indexing stays parallel) Now 8 concurrent adds of one file -> a single doc_id, zero errors. Adds a real-thread regression test. Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS |
||
|---|---|---|
| .. | ||
| backend | ||
| index | ||
| parser | ||
| storage | ||
| __init__.py | ||
| agent.py | ||
| client.py | ||
| cloud_api.py | ||
| collection.py | ||
| config.py | ||
| errors.py | ||
| events.py | ||
| page_index.py | ||
| page_index_md.py | ||
| retrieve.py | ||
| tokens.py | ||
| types.py | ||
| utils.py | ||