webclaw/crates/webclaw-core
Valerio 7f0420bbf0
fix(core): UTF-8 char boundary panic in find_content_position (#16) (#24)
`search_from = abs_pos + 1` landed mid-char when a rejected match
started on a multi-byte UTF-8 character, panicking on the next
`markdown[search_from..]` slice. Advance by `needle.len()` instead —
always a valid char boundary, and skips the whole rejected match
instead of re-scanning inside it.

Repro: webclaw https://bruler.ru/about_brand -f json
Before: panic "byte index 782 is not a char boundary; it is inside 'Ч'"
After:  extracts 2.3KB of clean Cyrillic markdown with 7 sections

Two regression tests cover multi-byte rejected matches and
all-rejected cycles in Cyrillic text.

Closes #16

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 12:02:52 +02:00
..
src fix(core): UTF-8 char boundary panic in find_content_position (#16) (#24) 2026-04-17 12:02:52 +02:00
testdata fix: prevent stack overflow on deeply nested HTML pages 2026-04-03 23:45:19 +02:00
Cargo.toml feat: v0.1.4 — QuickJS integration for inline JavaScript data extraction 2026-03-26 10:28:16 +01:00