fix: address xhigh code-review findings on 2d46d68..8f536cb

Verified 12 findings from an xhigh-effort review of the prior review-fix
batch; all confirmed real. Most trace back to one root cause: the
build_index() text-stripping fix (8f536cb) correctly stopped Markdown from
leaking full text by default, but broke every path that assumed text could
be re-read later.

Correctness:
- LocalBackend._fill_node_text (get_document(include_text=True)) only handled
  PDF's start_index/end_index convention; Markdown nodes use line_num and got
  silently empty text. Now handles both.
- get_page_content's Markdown fallback (triggered when a StorageEngine
  legitimately returns None from get_pages()) read from the now-text-stripped
  structure. It now re-derives from the source file, mirroring the PDF
  fallback, so it no longer depends on structure text at all.
- add_document's PDF-only text-stripping branch (with the stale "markdown
  needs text in structure for fallback retrieval" comment) is now dead/wrong
  since build_index() already applies if_add_node_text uniformly — removed.
- _validate_llm_provider's keyless-provider allowlist was missing several
  local LiteLLM providers (xinference, llamafile, triton, oobabooga,
  openai_like, docker_model_runner, custom, custom_openai, petals) that need
  no API key just like ollama/lm_studio; expanded.
- The three agent-tool closures (get_document, get_document_structure,
  get_page_content) had three different not-found patterns; two bypassed the
  backend's DocumentNotFoundError entirely. Extracted LocalBackend.
  _require_document as the single existence check every method/tool now uses.
- examples/agentic_vectorless_rag_demo.py's hand-rolled Agent() didn't apply
  the litellm/ prefix normalization the SDK does internally, so its own
  documented "any LiteLLM provider" claim broke for non-openai models.
- cloud delete_collection's cache eviction removed the "folders unavailable"
  None sentinel too, forcing a wasted re-fetch; now only pops on a real id.

Cleanup / altitude:
- build_index() skips the remove_structure_text walk entirely when text was
  never added (content_based + if_add_node_summary=False + if_add_node_text=
  False) instead of a guaranteed no-op tree walk.
- page_index()'s locals()-capture-as-kwargs (fragile by construction) replaced
  with an explicit dict of the named parameters.
- run_pageindex.py's _cli_bool and the page_index_md.py legacy shim's
  _coerce_bool were duplicate, diverging implementations; both now bind
  directly to the canonical pageindex.index.page_index_md._coerce_bool.
- retrieve.py's _get_md_page_content delegated its own traversal instead of
  calling the canonical get_md_page_content; now a one-line delegation.
- FileTypeError's docstring now calls out the except-ordering gotcha from
  also subclassing ValueError.

17 new regression tests (tests/test_review_fixes_2.py) plus 2 updated in
tests/test_legacy_shims.py for the simplified md_to_tree shim. Full suite:
210 passed, 2 skipped.

Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
This commit is contained in:
mountain 2026-07-08 21:56:41 +08:00
parent 8f536cb8d7
commit 04cb9cb02d
12 changed files with 333 additions and 124 deletions

View file

@ -58,6 +58,16 @@ Answer based only on tool output. Be concise.
"""
def _normalize_model_for_agents_sdk(model: str) -> str:
"""The OpenAI Agents SDK only recognizes 'openai/' and 'litellm/' model
prefixes; route any other LiteLLM-style provider path (e.g. 'anthropic/...')
through litellm explicitly, mirroring what PageIndex itself does internally
for its built-in agent."""
if model and "/" in model and not model.startswith(("litellm/", "openai/")):
return f"litellm/{model}"
return model
def query_agent(col, doc_id: str, prompt: str, model: str, verbose: bool = False) -> str:
"""Run a document QA agent using the OpenAI Agents SDK.
@ -90,7 +100,7 @@ def query_agent(col, doc_id: str, prompt: str, model: str, verbose: bool = False
name="PageIndex",
instructions=AGENT_SYSTEM_PROMPT,
tools=[get_document, get_document_structure, get_page_content],
model=model,
model=_normalize_model_for_agents_sdk(model),
# model_settings=ModelSettings(reasoning={"effort": "low", "summary": "auto"}), # Uncomment to enable reasoning
)