mirror of
https://github.com/VectifyAI/PageIndex.git
synced 2026-07-15 21:11:05 +02:00
6c948a332b
The new SDK copied the legacy indexing pipeline into pageindex/index/ instead of moving it, leaving two divergent copies of page_index.py / page_index_md.py / utils.py. They had already drifted (the legacy copy still compared IndexConfig booleans against 'yes' — a separate fix), and every pipeline change had to be applied twice. Make pageindex/index/ the single source of truth (same pattern as the LegacyCloudAPI shim for the 0.2.x cloud SDK): - pageindex/index/utils.py absorbs the 27 legacy-only helpers/classes (get_page_tokens, convert_page_to_int, ConfigLoader, PDF text helpers, ...) so it's the sole utils module. Reconciled the diverged funcs: kept the modern versions, backported the #331 get_leaf_nodes .get() fix, and restored remove_fields' max_len parameter (superset). - index/page_index*.py now import `from .utils import *`; index/legacy_utils.py (a re-export of the old top-level utils) deleted. - Top-level page_index.py / page_index_md.py / utils.py become thin re-export shims that emit PendingDeprecationWarning. The md_to_tree shim coerces legacy 'yes'/'no' string flags to bool (the canonical version is boolean-typed). - ConfigLoader no longer reads the deleted config.yaml; it builds defaults from IndexConfig (was an unconditional FileNotFoundError). - __init__.py and retrieve.py import from pageindex.index.* directly so `import pageindex` does not trip the shims. Adds tests/test_legacy_shims.py pinning the contract: clean top-level import doesn't warn, legacy submodule imports warn, symbols still resolve, shim and canonical share one implementation, the #331 fix and ConfigLoader-without-yaml both hold, and the md_to_tree coercion works. Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
38 lines
1.3 KiB
Python
38 lines
1.3 KiB
Python
# pageindex/page_index_md.py
|
|
# Deprecation shim. The Markdown indexing pipeline now lives in
|
|
# pageindex/index/page_index_md.py (the single source of truth). This module
|
|
# re-exports it so legacy imports keep working.
|
|
#
|
|
# The canonical md_to_tree takes booleans; legacy callers passed 'yes'/'no'
|
|
# strings, so the wrapper below coerces them (a bare 'no' is otherwise truthy).
|
|
import warnings
|
|
|
|
warnings.warn(
|
|
"pageindex.page_index_md has moved to pageindex.index.page_index_md; "
|
|
"importing it from the top level is deprecated and will be removed in a "
|
|
"future release.",
|
|
PendingDeprecationWarning,
|
|
stacklevel=2,
|
|
)
|
|
|
|
from .index.page_index_md import * # noqa: F401,F403,E402
|
|
from .index.page_index_md import md_to_tree as _md_to_tree # noqa: E402
|
|
|
|
_BOOL_PARAMS = (
|
|
"if_thinning", "if_add_node_summary", "if_add_doc_description",
|
|
"if_add_node_text", "if_add_node_id",
|
|
)
|
|
|
|
|
|
def _coerce_bool(value):
|
|
if isinstance(value, str):
|
|
return value.strip().lower() in ("yes", "true", "1", "y", "on")
|
|
return bool(value)
|
|
|
|
|
|
async def md_to_tree(*args, **kwargs):
|
|
"""Legacy wrapper: coerce 'yes'/'no' string flags to bool, then delegate."""
|
|
for key in _BOOL_PARAMS:
|
|
if key in kwargs:
|
|
kwargs[key] = _coerce_bool(kwargs[key])
|
|
return await _md_to_tree(*args, **kwargs)
|