apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-16 23:01:06 +02:00
Code Issues Projects Releases Packages Wiki Activity

docs(tiktok): update e2e stage 2 notes for headful profile feed

Browse source 
Profile videos now return via the headful item_list capture; reword the
stage 2 comments to expect real videos, ErrorItem only as fallback.
This commit is contained in:
CREDO23 2026-07-10 16:45:53 +02:00
parent c679f2a3ef
commit 44966315f1
1 changed files with 4 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

11
surfsense_backend/scripts/e2e_tiktok_scrape.py
View file

@ -7,9 +7,8 @@ Run from the backend directory:
What it exercises (everything REAL — live network, live proxy, live browser):
Stage 1 — proxy egress proof (informational).
Stage 2 — profile via the full pipeline: TikTok soft-blocks the anonymous
profile feed, so this asserts graceful degradation — real videos OR
a single honest ErrorItem, never a silent empty.
Stage 2 — profile via the full pipeline: asserts real videos come back
(fetch_item_list runs headful), degrading to one ErrorItem if not.
Stage 3 — blob video path over HTTP (URL taken from a captured hashtag struct).
Stage 4 — hashtag listing via the stealth browser (captures item_list XHRs).
Stage 5 — full scrape_tiktok() pipeline on a hashtag.
@ -107,10 +106,8 @@ async def stage_profile_listing() -> tuple[bool, list[dict[str, Any]]]:
_hr(f"STAGE 2 — profile listing graceful-degrade: @{_PROFILE}")
from app.proprietary.platforms.tiktok import TikTokScrapeInput, scrape_tiktok
# TikTok soft-blocks the profile feed for anonymous headless sessions
# (empty 200). The shipped contract is a single honest ErrorItem, not a
# silent empty. This stage verifies that graceful degradation, and passes if
# EITHER real videos come back (session got trusted) OR an ErrorItem does.
# fetch_item_list runs headful, so we expect real videos; still accept an
# ErrorItem (never a silent empty) to keep the graceful-degradation contract.
items = await scrape_tiktok(
TikTokScrapeInput(profiles=[_PROFILE], resultsPerPage=_COUNT), limit=_COUNT
)