mirror of
https://github.com/VectifyAI/PageIndex.git
synced 2026-07-18 21:21:05 +02:00
refactor(sdk): typed returns, protocol contract, parser layering
Engineering-quality cleanups from the SDK review (no behavior change):
- Return-type discoverability: add pageindex/types.py with TypedDicts
(DocumentInfo, DocumentDetail, PageContent) and annotate Collection /
Backend methods with them; add docstrings to every public Collection
method (including the get_page_content `pages` spec). Exported from the
package. Zero runtime cost — these are plain dicts.
- Backend protocol as a real contract:
* query_stream is an async generator, so the protocol now declares it
as `def ... -> AsyncIterator[QueryEvent]` (not `async def`, which
typed it as a coroutine and never matched the implementations).
* custom-parser support is expressed as a runtime_checkable
SupportsParserRegistration capability protocol; the client uses
isinstance(...) instead of hasattr(...) duck-typing.
- Parser layering: move count_tokens into a leaf module pageindex/tokens.py
so parser/* imports it from there instead of reaching back into
pageindex.index (a reverse dependency). index.utils re-exports it for
backward compatibility.
Adds tests/test_architecture.py enforcing: parser never imports index,
count_tokens is a single shared leaf, the capability protocol works,
both backends satisfy Backend, and the TypedDicts are exported.
Claude-Session: https://claude.ai/code/session_01Kx5DgKbhK1N8autqXH8SmS
This commit is contained in:
parent
b3616f76a2
commit
b2756c73d2
10 changed files with 155 additions and 17 deletions
|
|
@ -1,7 +1,7 @@
|
|||
import re
|
||||
from pathlib import Path
|
||||
from .protocol import ContentNode, ParsedDocument
|
||||
from ..index.utils import count_tokens
|
||||
from ..tokens import count_tokens
|
||||
|
||||
|
||||
class MarkdownParser:
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import pymupdf
|
||||
from pathlib import Path
|
||||
from .protocol import ContentNode, ParsedDocument
|
||||
from ..index.utils import count_tokens
|
||||
from ..tokens import count_tokens
|
||||
|
||||
# Minimum image dimension to keep (skip icons/artifacts)
|
||||
_MIN_IMAGE_SIZE = 32
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue