mirror of
https://github.com/VectifyAI/PageIndex.git
synced 2026-07-24 21:41:04 +02:00
ccc8b43bb7
The prompt-injection hardening ported from main did not fit dev's direction
(graceful degradation + content-faithful, reasoning-based index). Adjust it:
- process_toc_no_page_numbers: a count-mismatched or reordered/renamed LLM
response no longer raises ValueError (which aborted the entire index at the
uncaught top-level path and defeated the return_exceptions degradation the
rest of the pipeline uses). Skip the untrusted chunk and continue; the
accuracy check falls back to another mode when too little gets filled.
- _secure_doc_text: drop the keyword blocklist that replaced phrases like
"act as"/"disregard" with [REDACTED]. Those occur in legitimate titles and
prose, so redaction corrupted document content. Keep the <user_document>
framing + _SYSTEM_HARDENING system instruction as the injection defense.
- _validate_chunk_physical_indices: tolerate non-list input (extract_json
returns {} on parse failure and may return a JSON object) instead of
crashing while iterating a dict.
- Remove _validate_physical_indices / _parse_physical_index: range validation
is already done by validate_and_truncate_physical_indices in meta_processor
and marker->int by convert_physical_index_to_int; the helpers duplicated both.
Tests updated to assert the skip-not-raise behavior and lock in no-redaction
and non-list tolerance. 314 passed, 2 skipped.
|
||
|---|---|---|
| .. | ||
| test_agent.py | ||
| test_architecture.py | ||
| test_client.py | ||
| test_cloud_backend.py | ||
| test_collection.py | ||
| test_concurrency.py | ||
| test_config.py | ||
| test_content_node.py | ||
| test_env_compat.py | ||
| test_errors.py | ||
| test_events.py | ||
| test_issue_163.py | ||
| test_legacy_sdk_contract.py | ||
| test_legacy_shims.py | ||
| test_legacy_utils_contract.py | ||
| test_local_backend.py | ||
| test_markdown_parser.py | ||
| test_page_content.py | ||
| test_page_index.py | ||
| test_page_index_md.py | ||
| test_pdf_parser.py | ||
| test_pipeline.py | ||
| test_review_fixes.py | ||
| test_review_fixes_2.py | ||
| test_review_fixes_3.py | ||
| test_review_fixes_4.py | ||
| test_sqlite_storage.py | ||
| test_storage_protocol.py | ||
| test_types.py | ||