Address PR #272 review:
- get_md_page_content / retrieve._get_md_page_content returned every node
whose line_num fell in [min(pages), max(pages)], so a non-contiguous spec
like "5,100" over-fetched everything in between. Match the exact requested
line numbers instead, mirroring the PDF path. (Same bug as #280.)
- Restore the CHATGPT_API_KEY -> OPENAI_API_KEY backward-compat alias dropped
when pageindex/utils.py became a re-export shim; users with only
CHATGPT_API_KEY set would otherwise fail auth after upgrading. It now runs
in __init__.py right after load_dotenv.
Adds regression tests for both.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
The new SDK copied the legacy indexing pipeline into pageindex/index/
instead of moving it, leaving two divergent copies of page_index.py /
page_index_md.py / utils.py. They had already drifted (the legacy copy
still compared IndexConfig booleans against 'yes' — a separate fix),
and every pipeline change had to be applied twice.
Make pageindex/index/ the single source of truth (same pattern as the
LegacyCloudAPI shim for the 0.2.x cloud SDK):
- pageindex/index/utils.py absorbs the 27 legacy-only helpers/classes
(get_page_tokens, convert_page_to_int, ConfigLoader, PDF text helpers,
...) so it's the sole utils module. Reconciled the diverged funcs:
kept the modern versions, backported the #331 get_leaf_nodes .get()
fix, and restored remove_fields' max_len parameter (superset).
- index/page_index*.py now import `from .utils import *`;
index/legacy_utils.py (a re-export of the old top-level utils) deleted.
- Top-level page_index.py / page_index_md.py / utils.py become thin
re-export shims that emit PendingDeprecationWarning. The md_to_tree
shim coerces legacy 'yes'/'no' string flags to bool (the canonical
version is boolean-typed).
- ConfigLoader no longer reads the deleted config.yaml; it builds
defaults from IndexConfig (was an unconditional FileNotFoundError).
- __init__.py and retrieve.py import from pageindex.index.* directly so
`import pageindex` does not trip the shims.
Adds tests/test_legacy_shims.py pinning the contract: clean top-level
import doesn't warn, legacy submodule imports warn, symbols still
resolve, shim and canonical share one implementation, the #331 fix and
ConfigLoader-without-yaml both hold, and the md_to_tree coercion works.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
* Consolidate tests/ into examples/documents/
* Add line_count and reorder structure keys
* Lazy-load documents with _meta.json index
* Update demo script and add pre-shipped workspace
* Extract shared helpers for JSON reading and meta entry building
* Add PageIndexClient with retrieve, streaming support and litellm integration
* Add OpenAI agents demo example
* Update README with example agent demo section
* Support separate retrieve_model configuration for index and retrieve
