mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-05-17 18:35:19 +02:00
chore: update E2E test documentation for clarity and local setup instructions
This commit is contained in:
Anish Sarkar
parent
68f45335bc
commit
5344fa47e6
2 changed files with 192 additions and 78 deletions
|
|
@ -1,48 +1,48 @@
|
|||
# Backend E2E Test Harness
|
||||
# Backend E2E Harness
|
||||
|
||||
Strict fakes + alternative entrypoints used **only** by Playwright E2E.
|
||||
Excluded from the production Docker image via `.dockerignore`.
|
||||
This directory contains the test-only backend entrypoints and fakes used by
|
||||
Playwright. They are not part of the production image: `.dockerignore` excludes
|
||||
`tests/`, and the E2E Docker stage copies this directory through a separate
|
||||
build context.
|
||||
|
||||
## Files
|
||||
|
||||
| Path | Role |
|
||||
| -------------------------------- | ------------------------------------------------------------------------------- |
|
||||
| `run_backend.py` | FastAPI entrypoint that hijacks `sys.modules` before importing `app.app:app` |
|
||||
| `run_celery.py` | Celery worker entrypoint with the same hijack + patch logic |
|
||||
| `middleware/scenario.py` | `X-E2E-Scenario` header → ContextVar (read by fakes) |
|
||||
| `fakes/composio_module.py` | Strict drop-in for the `composio` package; raises on unknown surface |
|
||||
| `fakes/llm.py` | `fake_get_user_long_context_llm` returning a `FakeListChatModel` |
|
||||
| `fakes/embeddings.py` | Deterministic 0.1-vector `embed_text` / `embed_texts` |
|
||||
| `fakes/fixtures/drive_files.json`| Canned Drive listings + file contents (incl. canary tokens) |
|
||||
| Path | Purpose |
|
||||
| --- | --- |
|
||||
| `run_backend.py` | Starts FastAPI after installing the test fakes into `sys.modules`. |
|
||||
| `run_celery.py` | Starts the Celery worker with the same fake setup. |
|
||||
| `middleware/scenario.py` | Reads `X-E2E-Scenario` into a request-scoped context var. |
|
||||
| `fakes/composio_module.py` | Fake `composio` package used by connector flows. |
|
||||
| `fakes/llm.py` | Fake chat model factory. |
|
||||
| `fakes/embeddings.py` | Deterministic embedding helpers. |
|
||||
| `fakes/fixtures/drive_files.json` | Drive fixture data and canary file contents. |
|
||||
|
||||
## Why a sys.modules hijack?
|
||||
## Why the import hook exists
|
||||
|
||||
Production code does `from composio import Composio` at module load
|
||||
time. By the time the FastAPI app object exists, that binding has
|
||||
already been resolved. The hijack runs **before** any `app.*` import,
|
||||
so the binding resolves to our strict fake. No production source
|
||||
changes; fakes are physically excluded from production images.
|
||||
Some production modules import SDK clients at module load time, for example
|
||||
`from composio import Composio`. By the time `app.app` has been imported, those
|
||||
bindings are already fixed.
|
||||
|
||||
Belt + suspenders + no internet: the strict `__getattr__` in every
|
||||
fake raises `NotImplementedError` if a future production code path
|
||||
introduces a new SDK call. CI also sets `HTTPS_PROXY=http://127.0.0.1:1`
|
||||
plus sentinel API keys so any leaked outbound HTTP fails immediately.
|
||||
The E2E entrypoints install fake modules in `sys.modules` before importing any
|
||||
`app.*` module. That lets the normal production code run while SDK calls resolve
|
||||
to local fakes.
|
||||
|
||||
## Adding a new fake
|
||||
The fakes should fail loudly. If production starts using a new SDK method that
|
||||
the fake does not implement, add that method to the fake instead of letting the
|
||||
test call the real service.
|
||||
|
||||
1. Create `fakes/<sdk>_module.py` modelled on `composio_module.py`.
|
||||
2. In `run_backend.py` and `run_celery.py`, register
|
||||
`sys.modules["<sdk>"] = _fake_<sdk>` before the `from app.app import app`
|
||||
line.
|
||||
3. If the new fake needs scenario branching, read from
|
||||
## Adding a fake
|
||||
|
||||
1. Add `fakes/<sdk>_module.py`.
|
||||
2. Register it in both `run_backend.py` and `run_celery.py` before importing
|
||||
`app.app` or `app.celery_app`.
|
||||
3. If the fake needs per-test behavior, read the current scenario from
|
||||
`tests.e2e.middleware.scenario.current_scenario()`.
|
||||
|
||||
## Reused by backend integration tests
|
||||
## Shared with backend integration tests
|
||||
|
||||
The strict fakes are not only for Playwright. Backend route integration
|
||||
tests can import the same fake before importing `app.app`, so Composio
|
||||
route tests exercise production route code without touching the real
|
||||
SDK:
|
||||
Backend integration tests can use the same fakes when they need production route
|
||||
code without the real SDK:
|
||||
|
||||
```python
|
||||
from tests.e2e.fakes import composio_module as _fake_composio
|
||||
|
|
@ -50,20 +50,93 @@ sys.modules["composio"] = _fake_composio
|
|||
from app.app import app
|
||||
```
|
||||
|
||||
See `surfsense_backend/tests/integration/composio/conftest.py` for the
|
||||
current pattern.
|
||||
See `surfsense_backend/tests/integration/composio/conftest.py` for the current
|
||||
pattern.
|
||||
|
||||
## Running locally
|
||||
|
||||
The recommended local flow runs only Postgres and Redis in Docker, and the
|
||||
backend + Celery worker on the host. No `.env` file is required: both
|
||||
entrypoints `setdefault` every variable they need (DB URL, Redis URL,
|
||||
sentinel API keys, etc.) to values that match `docker-compose.deps-only.yml`.
|
||||
|
||||
### One-time setup
|
||||
|
||||
From `surfsense_web/`:
|
||||
|
||||
```bash
|
||||
cd surfsense_backend
|
||||
pnpm install
|
||||
pnpm exec playwright install --with-deps chromium
|
||||
```
|
||||
|
||||
### Each run
|
||||
|
||||
**1. Bring up Postgres + Redis** from the repo root (the other deps-only
|
||||
services (SearXNG, Zero, pgAdmin) are not needed for E2E):
|
||||
|
||||
```bash
|
||||
docker compose -f docker/docker-compose.deps-only.yml up -d db redis
|
||||
```
|
||||
|
||||
**2. Start the backend** in `surfsense_backend/`, terminal A:
|
||||
|
||||
```bash
|
||||
uv sync
|
||||
uv run alembic upgrade head
|
||||
uv run python tests/e2e/run_backend.py
|
||||
# in a second shell:
|
||||
```
|
||||
|
||||
**3. Start the Celery worker** in `surfsense_backend/`, terminal B:
|
||||
|
||||
```bash
|
||||
uv run python tests/e2e/run_celery.py
|
||||
```
|
||||
|
||||
Then in `surfsense_web`:
|
||||
**4. Register the Playwright user**:
|
||||
|
||||
```bash
|
||||
pnpm test:e2e
|
||||
curl -X POST http://localhost:8000/auth/register \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"email":"e2e-test@surfsense.net","password":"E2eTestPassword123!"}'
|
||||
```
|
||||
|
||||
**5. Run Playwright** from `surfsense_web/`, terminal C:
|
||||
|
||||
```bash
|
||||
pnpm test:e2e # dev server (fast iteration)
|
||||
pnpm test:e2e:headed # show the browser
|
||||
pnpm test:e2e:ui # Playwright UI mode
|
||||
pnpm test:e2e:prod # build + start (matches CI exactly)
|
||||
```
|
||||
|
||||
`playwright.config.ts` and the run scripts share defaults, so this works on a
|
||||
fresh checkout. Set `PLAYWRIGHT_TEST_EMAIL`, `PLAYWRIGHT_TEST_PASSWORD`,
|
||||
`NEXT_PUBLIC_FASTAPI_BACKEND_URL`, or any backend env (e.g. `DATABASE_URL`)
|
||||
only when pointing tests at a different stack.
|
||||
|
||||
### Cleanup
|
||||
|
||||
```bash
|
||||
docker compose -f docker/docker-compose.deps-only.yml down
|
||||
```
|
||||
|
||||
Add `-v` to also wipe the Postgres volume.
|
||||
|
||||
### Hermetic alternative (matches CI)
|
||||
|
||||
To reproduce the CI environment exactly — backend and Celery in containers,
|
||||
network egress denied at L3 — replace steps 1–3 with:
|
||||
|
||||
```bash
|
||||
docker compose -f docker/docker-compose.e2e.yml up -d --build --wait
|
||||
```
|
||||
|
||||
Then run steps 4 (curl register) and 5 (`pnpm test:e2e:prod`) as above. Tear
|
||||
down with:
|
||||
|
||||
```bash
|
||||
docker compose -f docker/docker-compose.e2e.yml down -v --remove-orphans
|
||||
```
|
||||
|
||||
This builds the ~9 GB `surfsense-e2e-backend:local` image, so the deps-only
|
||||
flow above is faster for day-to-day development.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue