mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-07-15 00:02:11 +02:00
f106ae2103
The entity URI normalizer stripped every non-ASCII character via the
ASCII-only patterns `[^a-z0-9\-.]` and `[^a-z0-9\-]`. For non-Latin
entity names (e.g. Chinese), this deleted the entire name, so distinct
entities collapsed onto the same URI (".../<ontology>/-"), silently
merging unrelated nodes and breaking OntoRAG for non-English content.
Switch both filters to `[^\w\-.]` / `[^\w\-]`. In Python 3, `\w` is
Unicode-aware for str patterns, so non-ASCII letters (CJK, etc.) are
preserved while punctuation and symbols are still removed. ASCII
behaviour is unchanged: names are lowercased and underscores converted
to hyphens before the filter runs, so for ASCII input the patterns are
equivalent to the originals.
Adds tests covering ASCII regression guards and non-ASCII names/types,
including the core case that two distinct Chinese entities must not
collapse onto the same URI.
Co-authored-by: arthurxuwei <5179840+arthurxuwei@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| agent | ||
| bootstrap | ||
| chunking | ||
| config/service | ||
| cores | ||
| decoding | ||
| direct | ||
| embeddings | ||
| external | ||
| extract | ||
| flow | ||
| gateway | ||
| iam | ||
| librarian | ||
| metering | ||
| model | ||
| processing | ||
| prompt | ||
| query | ||
| reranker | ||
| retrieval | ||
| rev_gateway | ||
| storage | ||
| tables | ||
| template | ||
| tool_service | ||
| __init__.py | ||