mirror of
https://github.com/0xMassi/webclaw.git
synced 2026-07-07 04:42:10 +02:00
3c54bea300
Rescued from the stale perf/audit-fixes branch — the *perf-only* subset of that branch's big mixed commit, ported cleanly onto current main with byte-identical extraction output. - markdown: hoist the `img[alt]` / `a[href]` selectors out of the per-node noise path into `Lazy` statics (stop recompiling them per element). - extractors: single shared `og()` / `parse_og()` module replaces the per-field Open Graph re-scan duplicated across 7 vertical extractors (amazon, ebay, ecommerce, etsy, substack, trustpilot, youtube). Each vertical now does one pass. Raw-vs-unescaped behaviour preserved exactly. - core: gate the QuickJS VM on a cheap marker check (skip it entirely when the page has no JS-assigned data) and reuse the already-parsed document instead of re-parsing the HTML. - fetch: connection-pool tuning on the wreq client (connect_timeout, idle pool, max-idle-per-host, tcp keepalive) for connection reuse. Output-equivalence is covered by existing tests (amazon quot-entity, trustpilot title parse, ecommerce/youtube/etsy/substack og fallbacks) — all green. No new dependencies; no public API change. Deliberately EXCLUDED from this slice (separate concerns bundled in the original commit): the `#[non_exhaustive]` API-breaking changes, the LLM/PDF/ server reliability hardening (much already shipped in 0.6.8), the tooling (cargo-deny, release profile, MSRV), and the retry-loop dedup refactor (a code-cleanup with no runtime benefit — not worth churning client.rs for). Original work by the prior author on perf/audit-fixes; this re-applies only the performance subset onto main. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| amazon_product.rs | ||
| arxiv.rs | ||
| crates_io.rs | ||
| dev_to.rs | ||
| docker_hub.rs | ||
| ebay_listing.rs | ||
| ecommerce_product.rs | ||
| etsy_listing.rs | ||
| github_issue.rs | ||
| github_pr.rs | ||
| github_release.rs | ||
| github_repo.rs | ||
| hackernews.rs | ||
| huggingface_dataset.rs | ||
| huggingface_model.rs | ||
| instagram_post.rs | ||
| instagram_profile.rs | ||
| linkedin_post.rs | ||
| mod.rs | ||
| npm.rs | ||
| og.rs | ||
| pypi.rs | ||
| reddit.rs | ||
| shopify_collection.rs | ||
| shopify_product.rs | ||
| stackoverflow.rs | ||
| substack_post.rs | ||
| trustpilot_reviews.rs | ||
| woocommerce_product.rs | ||
| youtube_video.rs | ||