Commit graph

3 commits

Author SHA1 Message Date
mountain
cf7f5ce9bf fix: address PR #272 review findings (directly-fixable items)
Verified against current dev; the compat/behavior decisions (#7 api_key
semantics, #10 CLI flags, #11 doc-description default) are deferred.

Crashes:
- page_index(): snapshot args before importing IndexConfig — locals() was
  capturing the imported class and IndexConfig(extra='forbid') made every call
  raise ValidationError.
- process_none_page_numbers: pop('page', None) instead of del (a TOC item
  without 'page' raised KeyError mid-pipeline).
- pipeline._run_async: guard only the loop detection, not the run, so a real
  RuntimeError from the coroutine isn't masked as "asyncio.run() cannot be
  called from a running event loop".

Silent-wrong / robustness:
- LocalBackend.get_document_structure and the agent get_document /
  get_document_structure tools now surface a missing doc (raise / error-JSON)
  instead of returning empty, matching get_page_content and the cloud backend.
- cloud delete_collection drops the cached folder_id.
- cloud query raises on an empty collection instead of POSTing doc_id:[].
- LocalClient skips the API-key check for keyless providers (ollama, lm_studio,
  …) so keyless LiteLLM models aren't rejected at construction.

Compat / cleanup:
- md_to_tree coerces legacy 'yes'/'no' string flags (a bare 'no' was truthy).
- FileTypeError also subclasses ValueError (0.2.x raised ValueError).
- _validate_llm_provider no longer mutates global litellm.model_cost_map_url.
- __all__ re-includes legacy exports (page_index, md_to_tree, get_*).
- Rewrite examples/agentic_vectorless_rag_demo.py to the Collection API and use
  the in-repo attention.pdf (the old workspace=/client.index/client.documents
  API no longer exists).

Adds tests/test_review_fixes.py (10 regressions). Full suite: 189 passed.

Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
2026-07-08 18:56:51 +08:00
mountain
e5392836a4 fix(index): bound LLM concurrency safely, per-index and leak-free
Cap concurrent in-flight LLM calls during indexing via a shared
semaphore (bounded_gather), so a many-node document no longer schedules
one socket per node and exhausts the process fd limit (Errno 24).

Make the per-index max_concurrency override correct under concurrency:
- Scope IndexConfig(max_concurrency=...) to the build_index call via a
  ContextVar (max_concurrency_scope) instead of mutating a process
  global. A one-off value no longer sticks as the new default, and
  concurrent indexing of other documents isn't affected.
- Propagate the context through _run_async's worker-thread fallback so
  the override survives the sync-over-async thread hop.
- set_max_concurrency() stays as the explicit process-wide setter.

Also stop `from .utils import *` leaking a `config` name (SimpleNamespace
alias) that shadowed the real pageindex.config submodule for the
page_index modules; the alias is now `_config`.

Adds regression tests for cap enforcement, scope stickiness/isolation,
worker-thread propagation, and the config-namespace fix.

Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
2026-07-08 10:40:37 +08:00
Kylin
c7fe93bb56 feat: add PageIndex SDK with local/cloud dual-mode support (#207) 2026-04-08 20:21:58 +08:00