- clamp LLM-derived page indices in _get_text_of_pages and
get_text_of_pdf_pages_with_labels; dedupe get_text_of_pdf_pages
- guard _normalize_tree and folders/documents iterations against
explicit nulls in cloud API responses
- coerce cloud OCR page numbers to int before filtering in
get_page_content
- folder cache: raise on missing folder id instead of caching None;
stop caching name-not-found so later lookups can succeed
- defang doc_id in agent doc-context prompt
- append api-key hint to 401 errors (request, legacy and streaming
paths)
- remove stale legacy JSON workspace sample data unreadable by the
SQLite storage
- local get_agent_tools: `set(doc_ids) if doc_ids is not None else None` so
doc_ids=[] is a scope of nothing (reject all), not open mode. The public
query path already guarded []; this hardens direct callers.
- legacy get_tree: send summary=true/false (lowercase) instead of Python's
capitalized True/False, matching the modern CloudBackend and the API.
- markdown parser: track the opening fence character so a ```-fence isn't
closed by a ~~~ line (CommonMark), keeping '#'-lines inside it out of headings.
- dedupe the cloud base URL: single API_BASE in cloud_api, referenced by
CloudBackend and PageIndexClient (was three independent copies).
Regression tests for each.
delete_document unconditionally called response.json(), so a successful
DELETE that returns 200 with an empty body (the documented examples don't
consume one, and REST APIs commonly return no content for deletes) would
raise JSONDecodeError even though the document was already deleted. Return
{} when the response has no content, else parse the JSON body as before.
chat_completions(stream=True, stream_metadata=True) used stream_metadata
only to pick the raw dict-chunk parser locally, never adding it to the
request payload. The wire request didn't match the caller's intent and
relied on the server sending metadata chunks unconditionally. Forward the
flag (mirroring the modern CloudBackend, which always sends it) so the
request is correct and robust if the server ever gates metadata behind it.
Verified against the real API that the server currently emits block_metadata
regardless, so this is a latent-correctness fix, not a behavior change today.
For faithful 0.2.x cloud SDK drop-in compatibility, is_retrieval_ready
again swallows PageIndexAPIError and returns False (instead of raising),
so existing `while not is_retrieval_ready(...)` polling loops behave
exactly as before. Documented that this can loop forever on a permanent
error — that is the legacy contract; callers guard their own loops.
(The new SDK's own indexing path doesn't use this method — it polls
document status with a bounded 120-attempt cap — so the infinite-loop
risk is confined to legacy-SDK usage that already had it.)
Test updated to assert the swallow behavior.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
- SQLite dedup race: add UNIQUE(collection_name, file_hash) and switch
save_document to a plain INSERT. add_document now catches the
IntegrityError from a concurrent add of the same content, cleans up its
managed files, and returns the winning doc_id — instead of two doc_ids
for one file (each having paid for its own LLM indexing).
- PDF image paths: store the absolute path to each extracted image
instead of a path relative to the indexing process's cwd. The
 references broke as soon as a query ran from a different
directory.
- LegacyCloudAPI: URL-encode doc_id / retrieval_id path segments (added
_enc()), matching CloudBackend. An id containing '/', '?', '#' or a
space previously hit the wrong endpoint or produced a malformed URL.
Adds regression tests: UNIQUE enforcement, the add-race resolving to the
winner, absolute image paths, and encoded legacy URLs.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
- AgentRunner.run: offload to a worker-thread event loop when called
from inside a running loop (Jupyter, FastAPI handlers) — mirrors
pipeline._run_async; Runner.run_sync raised RuntimeError there.
- SQLiteStorage: create connections with check_same_thread=False so
close() can actually close connections created by worker threads.
Each thread still gets its own connection via threading.local; with
the default True those closes raised ProgrammingError (silently
swallowed) and leaked every worker connection.
- CloudBackend.query: non-streaming chat completions now use a 300s
timeout and a single attempt. The default 30s ReadTimeout fired
before generation finished and the retry loop re-billed the full
server-side retrieval + generation up to three times. _request gains
retries/timeout overrides; the exhausted-retry path also no longer
sleeps before raising.
- MarkdownParser: content before the first heading (abstract/preamble)
becomes a node instead of being silently dropped and unretrievable;
a file with no headings at all yields a single document node instead
of zero nodes (which pushed an empty page list into the pipeline).
- LegacyCloudAPI.is_retrieval_ready: API failures (revoked key, network
down) now propagate as PageIndexAPIError instead of reading as
"not ready", which turned polling loops into infinite loops.
Adds regression tests for each fix.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
- local delete_collection: validate the collection name before rmtree.
An unvalidated name like "../.." escaped files_dir and deleted
arbitrary directories (path traversal).
- legacy page_index(): restore the node_id/summary/text/description
enhancements. IndexConfig now carries booleans (pydantic coerces the
legacy 'yes'/'no' strings at the boundary), but page_index_main still
compared `opt.if_add_node_id == 'yes'` — always False — so every
enhancement was silently skipped for legacy-API callers. Conditions
now branch on the booleans, matching pageindex/index/page_index.py.
- LegacyCloudAPI._request: bound every request with a timeout
(30s, 120s read timeout for streamed responses) so a dead connection
can't hang legacy submit/poll/chat callers forever. The legacy
contract tests pinned the missing timeout; updated to pin its
presence instead.
Adds regression tests: path-traversal rejection, 'yes'/'no' -> bool
coercion, and timeout assertions.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
* feat:compatible with Pageindex SDK
* corner cases fixed
* fix: mock behavior of old SDK
* fix: close streaming response and warn on empty api_key
- LegacyCloudAPI: close response in `finally` for both _stream_chat_response
variants so abandoned iterators no longer leak the TCP connection.
- PageIndexClient: emit a warning instead of silently falling back to local
when api_key is the empty string, surfacing typical env-var-unset misconfig.
- FakeResponse: add close()/closed to match the real requests.Response API.
- Add unit coverage for stream close (both paths) and the empty-api_key warning.
- Add scripts/e2e_legacy_sdk.py to smoke-test the legacy SDK contract end-to-end
against api.pageindex.ai.
* chore: mark legacy SDK methods with @deprecated and docstring pointers
- Decorate the 12 PageIndexClient cloud-SDK compat methods with
@typing_extensions.deprecated(..., category=PendingDeprecationWarning):
- IDE/type-checkers render them with a strikethrough hint
- runtime warnings stay silent by default (no spam for existing callers),
surfaceable via `python -W default::PendingDeprecationWarning`
- Add a one-line docstring on each pointing to the Collection-based equivalent.
- Promote typing-extensions to a direct dependency (was transitive via litellm).
---------
Co-authored-by: XinyanZhou <xinyanzhou@XinyanZhoudeMacBook-Pro.local>
Co-authored-by: saccharin98 <xinyanzhou938@gmail.com>
Co-authored-by: mountain <kose2livs@gmail.com>
