Commit graph

6 commits

Author SHA1 Message Date
mountain
fe36e25773 fix: six P2 correctness/robustness cleanups from the SDK review
- legacy call_llm: open AsyncOpenAI via `async with` so the client (and
  its HTTP connection pool) is closed instead of leaked.
- LocalBackend.add_document: fail fast with CollectionNotFoundError when
  the collection doesn't exist, before the expensive parse + LLM index
  (previously the missing FK only tripped at save time, after paying for
  the LLM work). Also raise builtin FileNotFoundError for a missing path
  instead of FileTypeError (which now means only "unsupported extension").
- Collection.query(doc_ids=None): the empty-collection guard now always
  runs — previously it was skipped once PAGEINDEX_EXPERIMENTAL_MULTIDOC
  was set. A single list_documents call serves both the guard and the
  multi-doc warning (no separate call just to decide whether to warn).
- CloudBackend.query_stream: on early consumer break / GeneratorExit,
  signal the background SSE thread to stop and force-close the response,
  so it no longer drains the whole stream in the background.

Adds regression tests for each (client closed, fail-fast on unknown
collection, FileNotFoundError, empty-check under the multidoc env flag,
single list call, early-break thread stop).

Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
2026-07-07 11:34:13 +08:00
mountain
6a73279c0c fix: patch three critical defects from the SDK review
- local delete_collection: validate the collection name before rmtree.
  An unvalidated name like "../.." escaped files_dir and deleted
  arbitrary directories (path traversal).

- legacy page_index(): restore the node_id/summary/text/description
  enhancements. IndexConfig now carries booleans (pydantic coerces the
  legacy 'yes'/'no' strings at the boundary), but page_index_main still
  compared `opt.if_add_node_id == 'yes'` — always False — so every
  enhancement was silently skipped for legacy-API callers. Conditions
  now branch on the booleans, matching pageindex/index/page_index.py.

- LegacyCloudAPI._request: bound every request with a timeout
  (30s, 120s read timeout for streamed responses) so a dead connection
  can't hang legacy submit/poll/chat callers forever. The legacy
  contract tests pinned the missing timeout; updated to pin its
  presence instead.

Adds regression tests: path-traversal rejection, 'yes'/'no' -> bool
coercion, and timeout assertions.

Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
2026-07-07 10:05:38 +08:00
mountain
284ce005f5 fix(cloud): align cloud/local backend contracts and harden the HTTP layer
Fixes the cloud/local contract mismatches from the PR #272 review
(verified against the official API docs — the cloud API has no
folder/collection endpoints publicly, GET /docs supports limit<=100
with offset):

- query_stream: emit a terminal answer_done event with the full answer
  (same contract as the local backend); raise CloudAPIError instead of
  disguising HTTP errors as answer events; move the initial connect
  inside try so a connection failure can no longer strand the consumer
  awaiting a sentinel that never arrives
- _request: rewind file objects before retrying so a transient 5xx/429
  during upload no longer re-sends an empty multipart body; carry the
  HTTP status on CloudAPIError (status_code) and keep the last status
  in the max-retries error
- list_documents: paginate with limit/offset instead of a hard-coded
  limit=100, so >100-doc collections are no longer silently truncated
  (whole-collection queries rely on this list)
- folders: treat only 403/404 as "folders unavailable" (warned via
  warnings.warn instead of an invisible logger.warning, matched on
  status_code instead of a "403" substring); transient errors now
  propagate instead of being permanently cached as folder_id=None
- error taxonomy parity: cloud doc endpoints map HTTP 404 to
  DocumentNotFoundError; local get_document raises DocumentNotFoundError
  instead of returning {}; local delete_document raises on missing
  doc_id instead of silently deleting nothing
- cloud get_document warns that include_text is not supported instead
  of silently ignoring it

Adds regression tests for each fix (11 new tests).

Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
2026-07-07 09:40:09 +08:00
mountain
a47c36a3f5 feat(collection): doc_ids accepts str|list, design cleanups
- Collection.query and Backend.query/query_stream accept doc_ids as
  str, list[str] or None. Single str is normalized to [str] inside each
  backend; bare [] is rejected with ValueError at both layers.
- wrap_with_doc_context wraps the scoped doc list in <docs>...</docs>
  and SCOPED_SYSTEM_PROMPT instructs the agent to treat that block as
  data, not instructions (defense against prompt injection via
  auto-generated doc_description).
- _require_cloud_api now distinguishes api_key="" from api_key=None;
  the former gives a targeted error pointing at the empty-string vs
  fall-back-to-local situation when legacy SDK methods are called.
- Legacy PageIndexClient.list_documents docstring spells out the
  return-shape difference vs collection.list_documents() to flag a
  silent migration footgun (paginated dict with id/name keys vs plain
  list[dict] with doc_id/doc_name keys).
- Remove dead CloudBackend.get_agent_tools stub (not on the Backend
  protocol; only ever returned an empty AgentTools()) and the
  SYSTEM_PROMPT alias (OPEN_/SCOPED_SYSTEM_PROMPT are the explicit
  names now).
- README quick start and streaming example now pass doc_ids; new
  multi-document section shows both str and list forms.
- examples/demo_query_modes.py exercises all five query-mode cases
  (single-doc, multi-doc with/without env var, scoped single, scoped
  multi) for manual verification.
2026-05-15 17:03:17 +08:00
mountain
d7b36aaf3f feat(collection): scoped query mode and experimental multi-doc warning
- get_agent_tools branches on doc_ids:
  - scoped (doc_ids=[...]): drops list_documents and hard-enforces a
    whitelist on the remaining tools; system prompt switches to
    SCOPED_SYSTEM_PROMPT (no list_documents instruction); doc list +
    summaries are prepended to the user message via wrap_with_doc_context.
  - open (doc_ids=None): unchanged 4-tool agent loop.
- list_documents now exposes doc_description (sqlite + cloud).
- Collection.query emits UserWarning when doc_ids is None and the
  collection holds >1 documents; PAGEINDEX_EXPERIMENTAL_MULTIDOC=1
  silences it. Single-doc collections skip the warning; empty
  collections raise ValueError.
- Agents SDK tracing upload disabled by default (avoids SSL timeouts);
  PAGEINDEX_AGENTS_TRACING=1 re-enables it.
- README: new SDK Usage section covering local/cloud quick start,
  streaming, multi-doc as experimental, and runnable examples.
2026-05-15 11:14:12 +08:00
Kylin
c7fe93bb56 feat: add PageIndex SDK with local/cloud dual-mode support (#207) 2026-04-08 20:21:58 +08:00