mirror of
https://github.com/VectifyAI/PageIndex.git
synced 2026-07-15 21:11:05 +02:00
e5392836a4
Cap concurrent in-flight LLM calls during indexing via a shared semaphore (bounded_gather), so a many-node document no longer schedules one socket per node and exhausts the process fd limit (Errno 24). Make the per-index max_concurrency override correct under concurrency: - Scope IndexConfig(max_concurrency=...) to the build_index call via a ContextVar (max_concurrency_scope) instead of mutating a process global. A one-off value no longer sticks as the new default, and concurrent indexing of other documents isn't affected. - Propagate the context through _run_async's worker-thread fallback so the override survives the sync-over-async thread hop. - set_max_concurrency() stays as the explicit process-wide setter. Also stop `from .utils import *` leaking a `config` name (SimpleNamespace alias) that shadowed the real pageindex.config submodule for the page_index modules; the alias is now `_config`. Adds regression tests for cap enforcement, scope stickiness/isolation, worker-thread propagation, and the config-namespace fix. Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS |
||
|---|---|---|
| .. | ||
| test_agent.py | ||
| test_architecture.py | ||
| test_client.py | ||
| test_cloud_backend.py | ||
| test_collection.py | ||
| test_concurrency.py | ||
| test_config.py | ||
| test_content_node.py | ||
| test_errors.py | ||
| test_events.py | ||
| test_issue_163.py | ||
| test_legacy_sdk_contract.py | ||
| test_legacy_shims.py | ||
| test_legacy_utils_contract.py | ||
| test_local_backend.py | ||
| test_markdown_parser.py | ||
| test_pdf_parser.py | ||
| test_pipeline.py | ||
| test_sqlite_storage.py | ||
| test_storage_protocol.py | ||