asyncio.to_thread(ceiling_sem.acquire) blocks a worker thread that
can't be interrupted. If the awaiting coroutine is cancelled (Ctrl-C,
an outer timeout) while that thread is still parked inside acquire(),
the thread can go on to actually acquire the permit after the
coroutine has already unwound — leaking it forever, since the
matching finally: release() never runs for that attempt. Poll with
the non-blocking acquire(False) form instead, which returns instantly
and closes the leak window entirely.
Addresses items 4-8 from the max-effort review of PR #272 (VectifyAI/PageIndex#272).
- agent.py: wrap_with_doc_context() strips '<'/'>' from doc_name/doc_description
(untrusted: unsanitized filename / LLM-generated from document content) before
inserting them into the <docs>...</docs> block, so embedded content can never
form a literal </docs> that closes the delimiter early and escapes the
untrusted-data boundary SCOPED_SYSTEM_PROMPT relies on. Deterministic
per-field transform, doesn't touch the (cacheable) static system prompt.
- ConfigLoader.load() (legacy 0.2.x compat) now routes merged overrides through
IndexConfig before returning, so a legacy 'no' string gets pydantic's bool
coercion instead of surviving as a truthy non-empty string — page_index_main's
bare `if opt.if_add_node_summary:` checks (changed from `== 'yes'` elsewhere
in this PR) were silently inverting caller intent and firing unwanted billed
LLM calls.
- verify_toc, process_large_node_recursively, tree_parser,
generate_summaries_for_structure, generate_summaries_for_structure_md: added
return_exceptions=True to their asyncio.gather calls (llm_completion/
llm_acompletion raise RuntimeError on retry exhaustion, added earlier in this
PR), each with a degrade path matching the pattern already used by sibling
hardened gathers in the same files. One transient LLM failure no longer
aborts the whole document's indexing.
- _llm_semaphore is now a true process-wide ceiling (threading.Semaphore,
shared across every thread/event loop) instead of one asyncio.Semaphore per
event loop -- concurrently indexing N documents on N threads no longer
multiplies the effective cap by N. A max_concurrency_scope() override is
layered as a second, nested, per-loop restriction that can only tighten the
effective cap within the ceiling, never widen past it.
- set_llm_params() mutated a bare process-wide dict with no per-call isolation,
unlike max_concurrency which already had ContextVar scoping. Added
llm_params_scope() (mirrors max_concurrency_scope) + IndexConfig.llm_params,
wired into build_index() the same way max_concurrency already was, so
concurrent indexing jobs with different llm kwargs don't leak into each
other.
Adds regression tests for all five. Full suite: 221 passed, 2 skipped (one
pre-existing, unrelated flaky cloud-streaming test intermittently fails on
rerun; confirmed independent of this change).
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
Address PR #272 review:
- get_md_page_content / retrieve._get_md_page_content returned every node
whose line_num fell in [min(pages), max(pages)], so a non-contiguous spec
like "5,100" over-fetched everything in between. Match the exact requested
line numbers instead, mirroring the PDF path. (Same bug as #280.)
- Restore the CHATGPT_API_KEY -> OPENAI_API_KEY backward-compat alias dropped
when pageindex/utils.py became a re-export shim; users with only
CHATGPT_API_KEY set would otherwise fail auth after upgrading. It now runs
in __init__.py right after load_dotenv.
Adds regression tests for both.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
The previous approach (bounded_gather building a fresh semaphore per call)
did NOT compose: the indexing call graph nests gathers
(tree_parser -> process_large_node_recursively -> recurse, plus each node's
check_title gather), and each level got its own independent cap. Peak
in-flight LLM calls grew ~N^depth, so a deep/wide document still exhausted
file descriptors (Errno 24) — the exact failure the cap was meant to
prevent — while the flat summary phase was over-serialized. Naively sharing
one semaphore across gather levels would instead deadlock (a parent holds a
slot while awaiting children that need slots).
Move the throttle to the single chokepoint every LLM call funnels through,
llm_acompletion: one shared semaphore per event loop, acquired only around
the litellm.acompletion network call. This gives a true global cap that
composes across any nesting and can't deadlock (a parent awaiting children
holds no slot). bounded_gather is gone; the call sites revert to plain
asyncio.gather.
Also:
- Reject bool in max_concurrency validation (bool is an int subclass, so
set_max_concurrency(True) / IndexConfig(max_concurrency=True) previously
became Semaphore(1) and silently serialized). Shared _validate_max_concurrency
+ a pydantic field_validator.
- Guard check_title_appearance_in_start_concurrent against an out-of-range
or 0 physical_index (LLM can emit one): it was dereferenced during task
construction, outside the gather's return_exceptions protection, aborting
the whole build; 0 silently wrapped to the last page. Now marked 'no'.
- Propagate contextvars into agent.py's worker-thread run (mirrors
pipeline._run_async) so ContextVar settings stay consistent.
- Tests rewritten to cover the nested case the old flat tests missed, the
leaf-level throttle in llm_acompletion, bool rejection, and the
out-of-range physical_index guard.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
Cap concurrent in-flight LLM calls during indexing via a shared
semaphore (bounded_gather), so a many-node document no longer schedules
one socket per node and exhausts the process fd limit (Errno 24).
Make the per-index max_concurrency override correct under concurrency:
- Scope IndexConfig(max_concurrency=...) to the build_index call via a
ContextVar (max_concurrency_scope) instead of mutating a process
global. A one-off value no longer sticks as the new default, and
concurrent indexing of other documents isn't affected.
- Propagate the context through _run_async's worker-thread fallback so
the override survives the sync-over-async thread hop.
- set_max_concurrency() stays as the explicit process-wide setter.
Also stop `from .utils import *` leaking a `config` name (SimpleNamespace
alias) that shadowed the real pageindex.config submodule for the
page_index modules; the alias is now `_config`.
Adds regression tests for cap enforcement, scope stickiness/isolation,
worker-thread propagation, and the config-namespace fix.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
Engineering-quality cleanups from the SDK review (no behavior change):
- Return-type discoverability: add pageindex/types.py with TypedDicts
(DocumentInfo, DocumentDetail, PageContent) and annotate Collection /
Backend methods with them; add docstrings to every public Collection
method (including the get_page_content `pages` spec). Exported from the
package. Zero runtime cost — these are plain dicts.
- Backend protocol as a real contract:
* query_stream is an async generator, so the protocol now declares it
as `def ... -> AsyncIterator[QueryEvent]` (not `async def`, which
typed it as a coroutine and never matched the implementations).
* custom-parser support is expressed as a runtime_checkable
SupportsParserRegistration capability protocol; the client uses
isinstance(...) instead of hasattr(...) duck-typing.
- Parser layering: move count_tokens into a leaf module pageindex/tokens.py
so parser/* imports it from there instead of reaching back into
pageindex.index (a reverse dependency). index.utils re-exports it for
backward compatibility.
Adds tests/test_architecture.py enforcing: parser never imports index,
count_tokens is a single shared leaf, the capability protocol works,
both backends satisfy Backend, and the TypedDicts are exported.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
- legacy call_llm: open AsyncOpenAI via `async with` so the client (and
its HTTP connection pool) is closed instead of leaked.
- LocalBackend.add_document: fail fast with CollectionNotFoundError when
the collection doesn't exist, before the expensive parse + LLM index
(previously the missing FK only tripped at save time, after paying for
the LLM work). Also raise builtin FileNotFoundError for a missing path
instead of FileTypeError (which now means only "unsupported extension").
- Collection.query(doc_ids=None): the empty-collection guard now always
runs — previously it was skipped once PAGEINDEX_EXPERIMENTAL_MULTIDOC
was set. A single list_documents call serves both the guard and the
multi-doc warning (no separate call just to decide whether to warn).
- CloudBackend.query_stream: on early consumer break / GeneratorExit,
signal the background SSE thread to stop and force-close the response,
so it no longer drains the whole stream in the background.
Adds regression tests for each (client closed, fail-fast on unknown
collection, FileNotFoundError, empty-check under the multidoc env flag,
single list call, early-break thread stop).
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
The new SDK copied the legacy indexing pipeline into pageindex/index/
instead of moving it, leaving two divergent copies of page_index.py /
page_index_md.py / utils.py. They had already drifted (the legacy copy
still compared IndexConfig booleans against 'yes' — a separate fix),
and every pipeline change had to be applied twice.
Make pageindex/index/ the single source of truth (same pattern as the
LegacyCloudAPI shim for the 0.2.x cloud SDK):
- pageindex/index/utils.py absorbs the 27 legacy-only helpers/classes
(get_page_tokens, convert_page_to_int, ConfigLoader, PDF text helpers,
...) so it's the sole utils module. Reconciled the diverged funcs:
kept the modern versions, backported the #331 get_leaf_nodes .get()
fix, and restored remove_fields' max_len parameter (superset).
- index/page_index*.py now import `from .utils import *`;
index/legacy_utils.py (a re-export of the old top-level utils) deleted.
- Top-level page_index.py / page_index_md.py / utils.py become thin
re-export shims that emit PendingDeprecationWarning. The md_to_tree
shim coerces legacy 'yes'/'no' string flags to bool (the canonical
version is boolean-typed).
- ConfigLoader no longer reads the deleted config.yaml; it builds
defaults from IndexConfig (was an unconditional FileNotFoundError).
- __init__.py and retrieve.py import from pageindex.index.* directly so
`import pageindex` does not trip the shims.
Adds tests/test_legacy_shims.py pinning the contract: clean top-level
import doesn't warn, legacy submodule imports warn, symbols still
resolve, shim and canonical share one implementation, the #331 fix and
ConfigLoader-without-yaml both hold, and the md_to_tree coercion works.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
llm_completion / llm_acompletion (pageindex/utils.py and
pageindex/index/utils.py) set `litellm.drop_params = True` on the litellm
module. litellm is a process-wide singleton, so this leaked into every other
library sharing it (e.g. a host app like OpenKB that exposes its own litellm
config) and could not be turned off.
Replace the hardcoded `temperature=0` + global `drop_params` with a single
PageIndex-owned per-call kwargs mechanism (config._LLM_PARAMS):
- defaults preserve behavior: {"temperature": 0, "drop_params": True};
- passed per call via **get_llm_params(), never writing litellm's globals, so
nothing leaks into other litellm users in the same process;
- externally configurable: pageindex.set_llm_params(drop_params=False,
temperature=1, num_retries=5, ...) or the PAGEINDEX_DROP_PARAMS env shortcut;
- model/messages are reserved (PageIndex supplies them) and rejected.
