apunkt/mike
1
0
Fork 0
mirror of https://github.com/willchen96/mike.git synced 2026-07-26 23:51:08 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

test(e2e): drop transient-5xx retry scaffolding now that the fetch storm is fixed

Browse source 
With the directory fetch storm eliminated (previous commit: batched
listProjects?include=documents instead of a getProject burst per modal
open), the local Supabase gateway no longer collapses under modal-open
load, so the specs' 5xx-oriented scaffolding is removed:

- create-with-retry loops (project create x2, chat create x2, workflow
  create, tabular-review create x10-attempt, review add-document x6)
- reload-guards for transient "Project not found" (waitForProjectLoaded,
  gotoProjectRow, project-assistant tab loop)
- networkidle settle-waits whose purpose was to let the per-project fetch
  burst drain before submitting (NewProjectModal / NewTRModal)
- oversized per-test timeouts justified by the storm (180s -> 60-120s)

Kept, deliberately:
- the title-generation settle logic in chat rename/delete
- chat submit's model-seeding retry loop (guards a documented
  useSelectedModel localStorage hydration race, unrelated to the gateway)
- account display-name toPass block (guards a documented profile
  hydration race, unrelated to the gateway)
- generic short networkidle settles after login/logout navigation
- Playwright's own retries: 2 on CI

Verified: with the fix, 3 consecutive full-suite runs (plus 2 more before
de-retry) were green with zero Playwright retries, running against an
account seeded with ~40 projects — a larger directory fan-out than CI's
fresh database ever produces.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
QA Runner 2026-07-20 11:44:00 -07:00
parent 5f8e4ab4d8
commit af38a45067
5 changed files with 169 additions and 503 deletions
Download patch file Download diff file Expand all files Collapse all files

153
e2e/chat-management.spec.ts
View file

@ -219,17 +219,9 @@ test("delete chat: sidebar delete action removes the chat from history", async (
await page.goto("/assistant");
const textarea = page.getByPlaceholder("How can I help?");
await expect(textarea).toBeVisible({ timeout: 10_000 });
// ── Step 2: create the chat, riding out transient gateway 502s ───────────────
// ── Step 2: create the chat ──────────────────────────────────────────────────
// ChatInput.handleSubmit creates the chat (saveChat → POST /chat/create) then
// navigates to /assistant/chat/<id>. The local Supabase/Kong gateway
// intermittently returns 502 on that POST under load; saveChat then returns null
// (createChat throws → caught in ChatHistoryContext.saveChat), so no chat is
// created and no navigation happens — the page stays on /assistant with the
// textarea intact, so re-submitting recovers the transient failure. This mirrors
// the bounded create-with-retry the project-assistant test (Step 7-8 below) uses
// for the same flaky POST. A genuinely broken create/navigation never reaches
// /assistant/chat/<id> on any attempt, so the final waitForURL still fails and
// the regression this test guards is preserved.
// navigates to /assistant/chat/<id>.
const newChatUrl = /\/assistant\/chat\/.+/;
// Sending the first message kicks off auto title-generation
@ -246,22 +238,11 @@ test("delete chat: sidebar delete action removes the chat from history", async (
)
.catch(() => null);
const CREATE_ATTEMPTS = 4;
for (let attempt = 0; attempt < CREATE_ATTEMPTS; attempt++) {
if (newChatUrl.test(page.url())) break;
// Re-assert the demo model as the active (available) model in case a
// remount reset it to the default Gemini (which the ApiKeyMissingModal
// would block on), then re-fill and re-submit.
await selectDemoModel(page);
await textarea.fill(message);
await textarea.press("Enter");
try {
await page.waitForURL(newChatUrl, { timeout: 20_000 });
break;
} catch {
// transient gateway 5xx on POST /chat/create — still on /assistant; retry
}
}
// Pick the keyless demo model so the submit isn't blocked by the
// ApiKeyMissingModal, then send the first message.
await selectDemoModel(page);
await textarea.fill(message);
await textarea.press("Enter");
await page.waitForURL(newChatUrl, { timeout: 20_000 });
// Let auto title-generation land before renaming, so it cannot clobber the
@ -296,22 +277,13 @@ test("delete chat: sidebar delete action removes the chat from history", async (
.locator("div.group.relative.h-8.rounded-md")
.filter({ has: targetTitle });
// ── Step 5-7: delete that specific chat, riding out flaky Supabase 500s ──────
// deleteChatFn (ChatHistoryContext.tsx:157-168) optimistically removes the row
// then, on API error, refetches via loadChats() which RESTORES it — so a
// transient 500 on DELETE /chat/<id> re-adds the row. Retry the open→Delete
// interaction until the uniquely-titled row is gone. A genuinely broken delete
// never removes it on any attempt, so the final assertion still fails.
// SidebarChatItem.tsx:132-144: the "Delete" DropdownMenuItem calls
// ── Step 5-7: delete that specific chat ──────────────────────────────────────
// deleteChatFn (ChatHistoryContext.tsx:157-168) optimistically removes the
// row. SidebarChatItem.tsx:132-144: the "Delete" DropdownMenuItem calls
// deleteChat(chat.id) directly — no confirmation dialog.
const DELETE_ATTEMPTS = 4;
for (let attempt = 0; attempt < DELETE_ATTEMPTS; attempt++) {
if (!(await targetTitle.isVisible().catch(() => false))) break;
await targetRow.hover();
await targetRow.locator("button").last().click();
await page.getByRole("menuitem", { name: "Delete" }).click();
await expect(targetTitle).toBeHidden({ timeout: 10_000 }).catch(() => {});
}
await targetRow.hover();
await targetRow.locator("button").last().click();
await page.getByRole("menuitem", { name: "Delete" }).click();
await expect(targetTitle).toBeHidden({ timeout: 10_000 });
});
@ -325,13 +297,10 @@ test("project assistant: create a new chat and submit a question", async ({ page
// removing that router.push: "+ Create New" then no longer navigates and the
// Step 8 waitForURL below fails.)
// This test creates a project then a chat (two sequential write round-trips plus
// a route compile/render each). Under a loaded dev server those round-trips can
// each take tens of seconds, so give the whole test — and the two create waits —
// generous headroom. Reliability matters more than speed here. The bounded
// create-with-retry on both the project and the chat (each up to a few
// re-submits under load) needs headroom beyond the create round-trips.
test.setTimeout(180_000);
// This test creates a project then a chat (two sequential write round-trips
// plus a navigation each) and ends with an LLM-backed submit, so give it
// headroom beyond the 30s default.
test.setTimeout(120_000);
// ── Step 1: navigate to projects ─────────────────────────────────────────────
await page.goto("/projects");
@ -354,105 +323,41 @@ test("project assistant: create a new chat and submit a question", async ({ page
// "Create project" submit button only exists on the second step.
await page.getByRole("button", { name: "Next", exact: true }).click();
// ── Step 4: submit the form (resilient to transient gateway 502s) ────────────
// ── Step 4: submit the form ──────────────────────────────────────────────────
// NewProjectModal.handleSubmit does NOT navigate itself — it calls onCreated()
// then onClose(). ProjectsOverview.onCreated (ProjectsOverview.tsx:475-478)
// inserts the row AND router.push(`/projects/${p.id}`), so the app navigates
// straight to the project detail page; the name never re-appears in a list to
// click. Wait for that navigation instead of asserting a list row.
//
// The local Supabase/Kong gateway intermittently returns 502 on POST /projects
// under load. On a failed create, handleSubmit catches the error, leaves the
// modal open with the name retained, and re-enables the submit button — so a
// transient failure is recovered by re-submitting. A genuine broken
// create/navigation never reaches /projects/<id> on any attempt, so the final
// assertion below still fails and that regression is preserved.
const submitBtn = page.getByRole("button", {
name: /create project|creating/i,
});
const projectUrl = /\/projects\/[^/]+$/;
const CREATE_ATTEMPTS = 4;
for (let attempt = 0; attempt < CREATE_ATTEMPTS; attempt++) {
if (projectUrl.test(page.url())) break;
// toBeEnabled rides out a slow in-flight "Creating…" from a prior attempt.
await expect(submitBtn).toBeEnabled({ timeout: 10_000 });
await submitBtn.click();
try {
await page.waitForURL(projectUrl, { timeout: 20_000 });
break;
} catch {
// transient gateway 5xx — modal stays open, name retained; retry
}
}
await expect(submitBtn).toBeEnabled({ timeout: 10_000 });
await submitBtn.click();
await page.waitForURL(projectUrl, { timeout: 20_000 });
// ── Step 6: open the assistant tab and reach the empty-state "+ Create New" ──
// ── Step 6: open the assistant tab and reach the empty-state "Create" ────────
// The project assistant is now a nested route (/projects/[id]/assistant), not a
// ?tab= query on the detail page. Navigating straight there avoids ambiguity
// with the sidebar "Assistant" nav item.
//
// The project workspace fetches getProject(id) on mount with no client-side
// refetch. Under load that GET can transiently 502, leaving project=null so the
// page renders "Project not found" and the assistant section never mounts. The
// project row genuinely exists (Step 4 navigated to its id), so a reload
// refetches and recovers. Bounded-retry the load until the empty-state
// "+ Create New" button (ProjectAssistantTable.tsx:110-115, shown when
// chats.length === 0) is visible, reloading past any transient "Project not
// found". A genuinely broken assistant tab never shows the button on any
// attempt, so the final assertion still fails.
const assistantUrl = page.url() + "/assistant";
// The olp UI replaced the old "+ Create New" text link with a PillButton
// reading "Create" in the ProjectAssistantTable empty state.
// reading "Create" in the ProjectAssistantTable empty state
// (ProjectAssistantTable.tsx:110-115, shown when chats.length === 0).
const createNewBtn = page.getByRole("button", { name: "Create", exact: true });
const projectNotFound = page.getByText("Project not found");
const TAB_ATTEMPTS = 4;
for (let attempt = 0; attempt < TAB_ATTEMPTS; attempt++) {
await page.goto(assistantUrl);
// Race the empty-state button against the transient "Project not found".
const outcome = await Promise.race([
createNewBtn
.waitFor({ state: "visible", timeout: 20_000 })
.then(() => "ready")
.catch(() => "retry"),
projectNotFound
.waitFor({ state: "visible", timeout: 20_000 })
.then(() => "notfound")
.catch(() => "retry"),
]);
if (outcome === "ready") break;
// "notfound" (transient getProject 502) or a timeout — reload to refetch.
}
await page.goto(assistantUrl);
await expect(createNewBtn).toBeVisible({ timeout: 20_000 });
// ── Step 7-8: click "+ Create New" and wait for the project chat URL ─────────
// ── Step 7-8: click "Create" and wait for the project chat URL ───────────────
// handleNewChat (ProjectPage.tsx:515-519) calls saveChat(projectId) then
// router.push(`/projects/${projectId}/assistant/chat/${id}`). On a transient
// gateway 502, saveChat returns null (createChat throws → caught in
// ChatHistoryContext.saveChat), so no chat is created and no navigation
// happens; the empty-state button stays mounted (chats still empty), so
// re-clicking recovers the transient failure.
// router.push(`/projects/${projectId}/assistant/chat/${id}`).
//
// REGRESSION (the target of this test): if handleNewChat's router.push is
// removed, saveChat still succeeds and adds the chat to state, so the
// empty-state "+ Create New" button is replaced by the chat list and no
// navigation occurs — the loop's visibility guard then stops retrying and the
// final waitForURL fails. If saveChat itself is broken, navigation never
// happens either. So a genuine break fails on every attempt and is preserved.
// removed — or saveChat itself is broken — no navigation happens and the
// waitForURL below fails.
const chatUrl = /\/projects\/.+\/assistant\/chat\/.+/;
const CHAT_ATTEMPTS = 4;
for (let attempt = 0; attempt < CHAT_ATTEMPTS; attempt++) {
if (chatUrl.test(page.url())) break;
// Empty-state button gone without navigation ⇒ a chat was created but
// never navigated (router.push regression): stop so it surfaces below.
if (!(await createNewBtn.isVisible().catch(() => false))) break;
await createNewBtn.click();
try {
await page.waitForURL(chatUrl, { timeout: 20_000 });
break;
} catch {
// transient gateway 5xx — empty-state remains; retry
}
}
await createNewBtn.click();
await page.waitForURL(chatUrl, { timeout: 20_000 });
// ── Step 9: assert the ChatInput textarea is visible ─────────────────────────

72
e2e/critical-path.spec.ts
View file

@ -55,11 +55,11 @@ test("create project, upload PDF, ask a question and receive a response", async
page,
}) => {
test.skip(!hasLlmKey, LLM_SKIP_REASON);
/* This end-to-end flow (create + upload + navigate + chat) is throttled by
the local Supabase stack and needs far more than the 30s default. The
per-test `{ timeout }` option that test() accepts is silently ignored by
Playwright (that object only takes tag/annotation), so set it here. */
test.setTimeout(180_000);
/* This end-to-end flow (create + upload + navigate + chat) needs more than
the 30s default. The per-test `{ timeout }` option that test() accepts is
silently ignored by Playwright (that object only takes tag/annotation),
so set it here. */
test.setTimeout(120_000);
/* ── Step 1: navigate to projects ─────────────────────────────────────── */
await page.goto("/projects");
@ -96,71 +96,29 @@ test("create project, upload PDF, ask a question and receive a response", async
).toBeVisible({ timeout: 5_000 });
/* ── Step 5: submit the form ──────────────────────────────────────────── */
/* The modal's FileDirectory (useDirectoryData) fires a burst of Supabase
requests when the modal opens — a getProject() per existing project.
Submitting mid-burst makes the local Supabase gateway (Kong) return a 502
("An invalid response was received from the upstream server"), surfaced
by the modal inline as text-red-500. Let those requests settle first so
the create POST goes through cleanly. */
await page
.waitForLoadState("networkidle", { timeout: 45_000 })
.catch(() => {});
/* The PDF upload runs inside NewProjectModal.handleSubmit
(await Promise.all([uploadProjectDocument(...)])) BEFORE onCreated fires,
so the "Creating…" button state can persist for many seconds while the
file uploads. ProjectsOverview.onCreated then router.push()es to the new
project page, so wait for that navigation (generously, to cover the
upload). Race it against the inline error so a residual transient 502 is
detected immediately and retried by re-submitting (safe — the failed
request created nothing). */
const inlineError = page.locator("form p.text-red-500");
for (let attempt = 1; attempt <= 5; attempt++) {
await page.click('button[type="submit"]');
const outcome = await Promise.race([
page
.waitForURL(/\/projects\/[^/]+$/, { timeout: 30_000 })
.then(() => "nav" as const)
.catch(() => "timeout" as const),
inlineError
.waitFor({ state: "visible", timeout: 30_000 })
.then(() => "error" as const)
.catch(() => "timeout" as const),
]);
if (outcome === "nav") break;
if (attempt === 5) {
throw new Error(
`create project: never navigated (last outcome: ${outcome})`,
);
}
await inlineError
.waitFor({ state: "hidden", timeout: 2_000 })
.catch(() => {});
}
upload). */
await page.click('button[type="submit"]');
await page.waitForURL(/\/projects\/[^/]+$/, { timeout: 30_000 });
/* ── Step 6: open the project assistant ───────────────────────────────── */
/* We're already on /projects/[id] (Documents tab by default). The project
assistant is now a nested route, /projects/[id]/assistant. Navigate there
directly rather than clicking through the tab bar to avoid ambiguity with
the "Assistant" item in the sidebar nav. The workspace fetches
getProject() on mount and does NOT retry, so under the local-Supabase load
the page can land on a permanent "Project not found" or a slow skeleton;
re-navigate until the assistant tab's empty-state "Create" affordance
renders. (The olp UI replaced the old "+ Create New" text link with a
PillButton reading "Create" — ProjectAssistantTable empty state.) */
the "Assistant" item in the sidebar nav. (The olp UI replaced the old
"+ Create New" text link with a PillButton reading "Create" —
ProjectAssistantTable empty state.) */
const projectUrl = page.url().split("?")[0];
const createNew = page.getByRole("button", { name: "Create", exact: true });
for (let attempt = 1; attempt <= 6; attempt++) {
await page.goto(`${projectUrl}/assistant`);
await page
.waitForLoadState("networkidle", { timeout: 20_000 })
.catch(() => {});
if (await createNew.isVisible().catch(() => false)) break;
}
await page.goto(`${projectUrl}/assistant`);
/* The assistant tab shows a chat list. Click "+ Create New" to open the
chat interface where the text input appears. */
await expect(createNew).toBeVisible({ timeout: 10_000 });
/* The assistant tab shows a chat list. Click "Create" to open the chat
interface where the text input appears. */
await expect(createNew).toBeVisible({ timeout: 20_000 });
await createNew.click();
/* Navigates to /projects/{id}/assistant (new chat UI) */
await page.waitForURL(/\/projects\/.+\/assistant/, { timeout: 10_000 });

138
e2e/project-management.spec.ts
View file

@ -33,18 +33,11 @@ async function createProject(
projectName: string,
filePath?: string,
) {
/* These tests are throttled by the local Supabase stack, which the app
hammers on every modal open (see the settle note below). The per-test
`{ timeout }` option passed to test() is silently ignored by Playwright
(that object only accepts tag/annotation), so the tests would otherwise
run at the 30s default — too tight for the directory storm. Raise it
here, where the slow work happens, so every caller gets the budget.
The storm scales with the number of projects the test user has
accumulated, and the modal now mounts FileDirectory on its second
("Add Documents") step, so creation alone can consume most of a 120s
budget and starve whatever the caller does afterwards. */
test.setTimeout(180_000);
/* Creation is a navigation + modal wizard + (optionally) a file upload; the
per-test `{ timeout }` option passed to test() is silently ignored by
Playwright (that object only accepts tag/annotation), so raise the budget
here, where the slow work happens, for every caller. */
test.setTimeout(60_000);
await page.goto("/projects");
await expect(page).toHaveURL(/\/projects/, { timeout: 10_000 });
@ -74,113 +67,48 @@ async function createProject(
).toBeVisible({ timeout: 5_000 });
}
/* The modal mounts a FileDirectory whose useDirectoryData hook fires several
Supabase-backed requests (listProjects + listStandaloneDocuments, plus a
getProject per project) the moment the modal opens. Submitting while those
are still in flight makes the local Supabase gateway (Kong) buckle under
the concurrent load and return a 502 — "An invalid response was received
from the upstream server" — which the API surfaces as a 500 and the modal
shows inline (text-red-500). Letting the modal's requests settle first
makes creation reliable (measured 6/6 vs 3/6 without this wait).
useDirectoryData fires getProject() for EVERY existing project, so this
storm grows with the project count and can take ~20s once many test
projects have accumulated. Bound the wait so we never hang on it; if it
doesn't fully settle, the submit retry below absorbs any residual 502. */
await page
.waitForLoadState("networkidle", { timeout: 45_000 })
.catch(() => {});
/* Submit — NewProjectModal's onCreated calls router.push(`/projects/${id}`).
The PDF upload runs (awaited) inside handleSubmit before onCreated fires,
so allow extra time for navigation when a file is attached.
Even with the settle above, a transient upstream 502 can still slip
through occasionally. NewProjectModal surfaces it inline (text-red-500)
while keeping the modal open and the form state intact, so re-clicking
submit retries safely (the failed request created nothing). Race the
navigation against that inline error so a failure is detected immediately
and retried, rather than burning the full navigation timeout each time. */
(The modal's FileDirectory used to fan out a getProject() request per
existing project on open, which could overwhelm the local Supabase
gateway and required settle-waits plus a submit-retry loop here. The
directory now loads via one batched listProjects?include=documents
request, so a single submit is reliable.)
The documents step's primary action submits the form (its label flips
to "Creating…" while in flight, so match on the submit type instead). */
const navTimeout = filePath ? 30_000 : 15_000;
const inlineError = page.locator("form p.text-red-500");
for (let attempt = 1; attempt <= 5; attempt++) {
/* The documents step's primary action submits the form (its label flips
to "Creating…" while in flight, so match on the submit role instead). */
await page.locator('button[type="submit"]').click();
const outcome = await Promise.race([
page
.waitForURL(/\/projects\/.+/, { timeout: navTimeout })
.then(() => "nav" as const)
.catch(() => "timeout" as const),
inlineError
.waitFor({ state: "visible", timeout: navTimeout })
.then(() => "error" as const)
.catch(() => "timeout" as const),
]);
if (outcome === "nav") return;
if (attempt === 5) {
throw new Error(
`createProject: never navigated to /projects/<id> (last outcome: ${outcome})`,
);
}
// transient upstream error (or stall) → wait for the inline message to
// clear so the next iteration's race can't latch onto the stale one,
// then resubmit.
await inlineError
.waitFor({ state: "hidden", timeout: 2_000 })
.catch(() => {});
}
await page.locator('button[type="submit"]').click();
await page.waitForURL(/\/projects\/.+/, { timeout: navTimeout });
}
/**
* Navigate to the projects list and return the table row for `projectName`,
* re-navigating (which refetches) if the list doesn't render it in time.
* ProjectsOverview gates its table on listProjects(); under the local-Supabase
* load that call can be slow or transiently 502 (the page then shows "Could not
* load projects."), so a single goto isn't reliable. Rows are <div class="group">;
* the sidebar "Recent Projects" renders the same name as a <button>, so scoping
* to div.group avoids a strict-mode double match.
* Navigate to the projects list and return the table row for `projectName`.
* Rows are <div class="group">; the sidebar "Recent Projects" renders the same
* name as a <button>, so scoping to div.group avoids a strict-mode double match.
*/
async function gotoProjectRow(
page: import("@playwright/test").Page,
projectName: string,
) {
const row = page.locator("div.group").filter({ hasText: projectName });
for (let attempt = 1; attempt <= 5; attempt++) {
await page.goto("/projects");
const shown = await row
.first()
.waitFor({ state: "visible", timeout: 12_000 })
.then(() => true)
.catch(() => false);
if (shown) return row;
}
await page.goto("/projects");
await expect(row.first()).toBeVisible({ timeout: 12_000 });
return row;
}
/**
* After creating a project we land on /projects/<id>, whose ProjectPage fetches
* getProject() once on mount and does NOT retry. Under the local-Supabase load
* that request can transiently 502 (rendering a permanent "Project not found")
* or simply be slow (leaving the loading skeleton up). Reload until `anchor` —
* a control that only renders once the project has loaded — becomes visible.
* getProject() once on mount. Wait until `anchor` — a control that only renders
* once the project has loaded — becomes visible.
*/
async function waitForProjectLoaded(
page: import("@playwright/test").Page,
anchor: import("@playwright/test").Locator,
) {
for (let attempt = 1; attempt <= 6; attempt++) {
const shown = await anchor
.waitFor({ state: "visible", timeout: 8_000 })
.then(() => true)
.catch(() => false);
if (shown) return;
if (attempt === 6) break;
await page.reload();
await page.waitForLoadState("domcontentloaded").catch(() => {});
}
await expect(anchor).toBeVisible({ timeout: 8_000 });
await expect(anchor).toBeVisible({ timeout: 15_000 });
}
// ─── Test 1: Rename a project inline ─────────────────────────────────────────
@ -190,10 +118,9 @@ test("rename a project via Edit details", async ({ page }) => {
await createProject(page, projectName);
/* Navigate to the projects list (where the rename UI lives) and grab the
row. Each project row is a <div class="group">; gotoProjectRow refetches
if the list is slow/errors under load. The sidebar "Recent Projects" list
renders the same name as a <button> (not div.group), so scoping to
div.group keeps the lookup to the table row. */
row. Each project row is a <div class="group">. The sidebar "Recent
Projects" list renders the same name as a <button> (not div.group), so
scoping to div.group keeps the lookup to the table row. */
const row = await gotoProjectRow(page, projectName);
/* The ··· button (middle-dot U+00B7 × 3) is inside the last cell of the row */
@ -236,7 +163,7 @@ test("delete a project", async ({ page }) => {
await createProject(page, projectName);
/*
* Back to the projects list (refetching if it's slow/errors under load).
* Back to the projects list.
* Rows are scoped to div.group — the sidebar "Recent Projects" list also
* shows the name as a <button>, so an unscoped getByText would match two
* elements.
@ -284,12 +211,8 @@ test("create a folder inside a project", async ({ page }) => {
/*
* After createProject we are on the new project page (Documents tab). The
* page fetches the project via getProject() on mount and does NOT retry; if
* that request transiently 502s under the local-Supabase load it renders a
* permanent "Project not found", and a slow getProject just leaves the page
* on its loading skeleton for a while. The documents toolbar (which hosts
* "Add Subfolder") only appears once the project has loaded, so reload until
* it does.
* documents toolbar (which hosts the folder-create button) only appears
* once the project has loaded.
*/
/* The olp UI renamed the documents-toolbar folder-create button from
"Add Subfolder" to "Folder" (a TabPillButton wired to the root
@ -348,9 +271,8 @@ test("file upload type validation — .txt file is rejected", async ({ page }) =
* endpoint directly with the browser session's bearer token.
*/
/* Open the Add Documents modal. The "Add Documents" button only renders once
ProjectPage has loaded the project; getProject() can transiently 502 (→
"Project not found") or be slow under load, so reload-guard the page. */
/* Open the Add Documents modal. The "Add Documents" button only renders
once ProjectPage has loaded the project. */
const addDocsBtn = page.getByRole("button", { name: "Add Documents" });
await waitForProjectLoaded(page, addDocsBtn);

184
e2e/tabular-reviews.spec.ts
View file

@ -13,13 +13,8 @@ import path from "path";
const PDF_FIXTURE = path.join(__dirname, "fixtures/test.pdf");
// Run these tests sequentially in a single worker. The global config sets
// fullyParallel:true, which would otherwise run the create/detail tests
// concurrently — and they each trigger the first on-demand `next dev` compile
// of the dynamic /tabular-reviews/[id] route at the same time. Under that
// compiler contention the dev server drops in-flight client navigations, so
// router.push() lands but the URL never changes. Serial mode lets the first
// test warm (compile) the route, after which the rest navigate near-instantly.
// Run these tests sequentially in a single worker: they share the one test
// user's review list, so concurrent create/detail runs would race each other.
test.describe.configure({ mode: "serial" });
/* ─── Helpers ────────────────────────────────────────────────────────────── */
@ -72,20 +67,6 @@ async function openNewReviewModal(page: import("@playwright/test").Page) {
* Create a tabular review through the real modal flow and land on its detail
* page. Returns the title that was entered so callers can assert on it.
*
* Two dev-environment flakes are tolerated here so the *behaviour under test*
* (create → detail page) is what's exercised, not infrastructure noise:
*
* 1. The local Supabase gateway (Kong) intermittently returns 500
* "An invalid response was received from the upstream server" / "fetch
* failed" when its PostgREST upstream is momentarily unavailable — observed
* on ~30% of create calls, and far more often while the modal's per-project
* fetch burst is hammering Supabase concurrently. We wait for that burst to
* settle before submitting, and retry the POST on transient 5xx. A genuinely
* broken create flow fails *every* attempt, so regressions are still caught.
* 2. `next dev` compiles the dynamic /tabular-reviews/[id] route on first
* navigation (15-30s cold); under that latency the dev server can drop the
* in-flight client navigation, so we fall back to an explicit goto.
*
* @param onFirstOpen optional assertion run against the modal on the first open
* (used by Test 2 to verify the workflow-template default renders).
*/
@ -100,64 +81,36 @@ async function createReview(
).toBeVisible({ timeout: 10_000 });
const reviewName = `${label} ${Date.now()}`;
let review: { id: string } | null = null;
for (let attempt = 0; attempt < 10 && !review; attempt++) {
if (attempt > 0) {
// The modal closes itself on submit; reopen for the retry.
await page.goto("/tabular-reviews");
await expect(
page.getByRole("heading", { name: "Tabular Reviews" }),
).toBeVisible({ timeout: 10_000 });
}
const titleInput = await openNewReviewModal(page);
if (onFirstOpen) await onFirstOpen();
await titleInput.fill(reviewName);
const titleInput = await openNewReviewModal(page);
if (attempt === 0 && onFirstOpen) await onFirstOpen();
await titleInput.fill(reviewName);
// NewTRModal is a two-step wizard ("Details" → "Add Documents"); the
// "Create" submit button only exists on the second step, and "Next" only
// enables once the review has a name.
await page.getByRole("button", { name: "Next", exact: true }).click();
// Let the modal's project-fetch burst settle so the create POST doesn't
// compete with it on the flaky local Supabase (best-effort; the HMR
// socket means networkidle may not fully settle, so it's time-boxed).
await page
.waitForLoadState("networkidle", { timeout: 8_000 })
.catch(() => {});
// exact: true hits the modal's submit button only — the list page's
// empty-state "+ Create New" CTA and the "Create under a project" toggle
// also contain the word "Create".
const respP = page
.waitForResponse(isCreateReviewPost, { timeout: 30_000 })
.catch(() => null);
// The modal footer's submit button and the page's own "Create" CTA both
// read "Create"; scope to the modal's submit (button[name="modalAction"]
// value="create-review") to avoid a strict-mode ambiguity.
await page
.locator('button[name="modalAction"][value="create-review"]')
.click();
const resp = await respP;
if (resp && resp.ok()) {
review = (await resp.json()) as { id: string };
}
// else: transient upstream 5xx — loop reopens the modal and retries.
}
// NewTRModal is a two-step wizard ("Details" → "Add Documents"); the
// "Create" submit button only exists on the second step, and "Next" only
// enables once the review has a name.
await page.getByRole("button", { name: "Next", exact: true }).click();
const respP = page.waitForResponse(isCreateReviewPost, {
timeout: 30_000,
});
// The modal footer's submit button and the page's own "Create" CTA both
// read "Create"; scope to the modal's submit (button[name="modalAction"]
// value="create-review") to avoid a strict-mode ambiguity.
await page
.locator('button[name="modalAction"][value="create-review"]')
.click();
const resp = await respP;
expect(
review,
"POST /tabular-review never returned 2xx after retries — create flow broken?",
).not.toBeNull();
resp.ok(),
`POST /tabular-review returned ${resp.status()} — create flow broken?`,
).toBe(true);
const review = (await resp.json()) as { id: string };
// createTabularReview() → router.push("/tabular-reviews/<id>").
await page
.waitForURL(`**/tabular-reviews/${review!.id}`, { timeout: 60_000 })
.catch(() => page.goto(`/tabular-reviews/${review!.id}`));
await expect(page).toHaveURL(
new RegExp(`/tabular-reviews/${review!.id}`),
{ timeout: 60_000 },
new RegExp(`/tabular-reviews/${review.id}`),
{ timeout: 30_000 },
);
return reviewName;
@ -188,12 +141,12 @@ test("navigates to /tabular-reviews and the list page renders", async ({
test("creates a new tabular review and is redirected to the detail page", async ({
page,
}) => {
// Headroom for the create-POST retries (flaky local Supabase) plus the
// on-demand `next dev` compile of the dynamic /tabular-reviews/[id] route.
test.setTimeout(180_000);
// Headroom for the create round-trip plus the first navigation to the
// dynamic /tabular-reviews/[id] route.
test.setTimeout(60_000);
// REGRESSION: fails if createTabularReview() API call is removed or the
// /tabular-reviews POST route is broken (every retry attempt then fails,
// so `review` stays null and createReview's not-null assertion trips).
// /tabular-reviews POST route is broken (createReview's response
// assertion then trips).
//
// createReview opens the modal, verifies the workflow-template default
// renders, submits, and lands on the new review's detail page.
@ -216,9 +169,8 @@ test("creates a new tabular review and is redirected to the detail page", async
test("review detail page renders the table structure and toolbar controls", async ({
page,
}) => {
// Headroom for create-POST retries (flaky local Supabase) plus the
// on-demand `next dev` compile of the dynamic detail route.
test.setTimeout(180_000);
// Headroom for the create round-trip plus the detail-route navigation.
test.setTimeout(60_000);
// REGRESSION: fails if the /tabular-reviews/[id] route, TRView, or TRTable
// component is broken
const reviewName = await createReview(page, "E2E Table Review");
@ -257,64 +209,40 @@ test("review detail page renders the table structure and toolbar controls", asyn
test("adds a document to a tabular review and the row appears in the table", async ({
page,
}) => {
// Headroom for create-POST retries, the detail-route compile, and the
// upload + document-link round-trips.
test.setTimeout(180_000);
// Headroom for the create round-trip plus the upload + document-link
// round-trips.
test.setTimeout(90_000);
// REGRESSION: fails if the document-to-review linking
// (PATCH /tabular-reviews/:id with document_ids) or the upload endpoint breaks
const reviewName = await createReview(page, "E2E Doc Review");
// reviewName is already confirmed visible on the detail page by createReview
// Add the document and assert the row appears. The upload endpoint and the
// document-link PATCH both go through the same flaky local Supabase that
// intermittently 500s (see createReview), so the whole open→upload→confirm
// round-trip is retried until the row renders. The behaviour under test —
// a successful upload + link surfacing the row — is unchanged; a genuine
// break in the upload or link path fails *every* attempt (the modal already
// re-deletes nothing, so each retry uploads a fresh copy).
const row = page.getByText("test.pdf").first();
const confirmBtn = page.getByRole("button", { name: "Confirm" });
for (let attempt = 0; attempt < 6; attempt++) {
// Open AddDocumentsModal (standalone path → AddDocumentsModal, not
// AddProjectDocsModal). first() handles both toolbar & empty-state CTA.
const addDocsBtn = page
.getByRole("button", { name: /Add Documents/ })
.first();
await expect(addDocsBtn).toBeVisible({ timeout: 10_000 });
await addDocsBtn.click();
// Open AddDocumentsModal (standalone path → AddDocumentsModal, not
// AddProjectDocsModal). first() handles both toolbar & empty-state CTA.
const addDocsBtn = page
.getByRole("button", { name: /Add Documents/ })
.first();
await expect(addDocsBtn).toBeVisible({ timeout: 10_000 });
await addDocsBtn.click();
// The footer's "Upload" button programmatically clicks a hidden
// <input type="file"> — Playwright intercepts it as a file-chooser event.
const uploadBtn = page.getByRole("button", { name: "Upload" });
await expect(uploadBtn).toBeVisible({ timeout: 5_000 });
const fileChooserPromise = page.waitForEvent("filechooser");
await uploadBtn.click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles(PDF_FIXTURE);
// The footer's "Upload" button programmatically clicks a hidden
// <input type="file"> — Playwright intercepts it as a file-chooser event.
const uploadBtn = page.getByRole("button", { name: "Upload" });
await expect(uploadBtn).toBeVisible({ timeout: 5_000 });
const fileChooserPromise = page.waitForEvent("filechooser");
await uploadBtn.click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles(PDF_FIXTURE);
// After a successful upload the server document is auto-selected and the
// "Confirm" button transitions disabled → enabled. A 5xx upload leaves it
// disabled — close the modal and retry.
const becameEnabled = await expect(confirmBtn)
.toBeEnabled({ timeout: 20_000 })
.then(() => true)
.catch(() => false);
if (!becameEnabled) {
await page.getByRole("button", { name: "Cancel" }).click();
continue; // upload 5xx'd — retry
}
// Confirm → onSelect() → handleAddDocuments() → updateTabularReview()
// PATCH → setDocuments() → TRTable renders the new row with doc.filename.
await confirmBtn.click();
const appeared = await row
.waitFor({ state: "visible", timeout: 15_000 })
.then(() => true)
.catch(() => false);
if (appeared) break;
// The link PATCH may have 5xx'd (modal already closed) — loop and retry.
}
// After a successful upload the server document is auto-selected and the
// "Confirm" button transitions disabled → enabled.
await expect(confirmBtn).toBeEnabled({ timeout: 20_000 });
// Confirm → onSelect() → handleAddDocuments() → updateTabularReview()
// PATCH → setDocuments() → TRTable renders the new row with doc.filename.
await confirmBtn.click();
await expect(row).toBeVisible({ timeout: 15_000 });
});

125
e2e/workflows-account.spec.ts
View file

@ -20,44 +20,20 @@ import { test, expect, type Page } from "@playwright/test";
/**
* Create a workflow from an already-open NewWorkflowModal and wait for the
* post-create navigation to /workflows/<id>.
*
* The local Supabase/Kong gateway intermittently returns 502 on POST /workflows
* under load. On a failed create the modal stays open with the entered name
* retained — NewWorkflowModal.handleSubmit only calls onCreated()/onClose() on
* success — so a transient failure is recovered by re-submitting the form.
*
* This retries ONLY transient failures: a genuine create regression (persistent
* 5xx) never navigates on any attempt, so the final assertion still fails and the
* regression is preserved.
*/
async function createWorkflowAndOpenDetail(page: Page, title: string) {
const nameInput = page.getByPlaceholder("Workflow name");
await expect(nameInput).toBeVisible({ timeout: 5_000 });
await nameInput.fill(title);
// Match the submit button in BOTH states: its label is "Create workflow" when idle
// and "Creating…" while a request is in flight. Matching only "Create workflow" would
// make the button "not found" mid-submit and break the retry loop.
// Match the submit button in BOTH states: its label is "Create workflow" when
// idle and "Creating…" while the request is in flight.
const createBtn = page.getByRole("button", {
name: /create workflow|creating/i,
});
const MAX_ATTEMPTS = 4;
for (let attempt = 0; attempt < MAX_ATTEMPTS; attempt++) {
// Already navigated (success on a prior attempt)? Done.
if (/\/workflows\/.+/.test(page.url())) return;
// Wait until the button is idle/enabled ("Create workflow"), then re-submit.
// toBeEnabled rides out a slow in-flight "Creating…" from the previous attempt.
await expect(createBtn).toBeEnabled({ timeout: 10_000 });
await createBtn.click();
try {
await expect(page).toHaveURL(/\/workflows\/.+/, { timeout: 10_000 });
return;
} catch {
// Transient gateway 5xx — modal stays open with the name retained; retry.
}
}
// Final assertion: surfaces a persistent (non-transient) create failure clearly.
await expect(page).toHaveURL(/\/workflows\/.+/, { timeout: 10_000 });
await expect(createBtn).toBeEnabled({ timeout: 10_000 });
await createBtn.click();
await expect(page).toHaveURL(/\/workflows\/.+/, { timeout: 15_000 });
}
/* ─────────────────────────────────────────────────────────────────────────────
@ -116,9 +92,8 @@ test.describe("Workflows", () => {
// Fill the title, submit, and wait for the post-create router.push to
// /workflows/<id>. Type defaults to "Assistant" — no change needed.
// The helper re-submits on transient gateway 502s (see top of file).
// REGRESSION: a broken workflow-create API never navigates on any attempt,
// so the helper's final toHaveURL assertion still fails.
// REGRESSION: a broken workflow-create API never navigates, so the
// helper's toHaveURL assertion fails.
const workflowTitle = `E2E Workflow ${Date.now()}`;
await createWorkflowAndOpenDetail(page, workflowTitle);
@ -180,10 +155,6 @@ test.describe("Workflows", () => {
await newWorkflowBtn.click();
const workflowTitle = `E2E Edit Workflow ${Date.now()}`;
// Resilient create: the inline workflow-create here intermittently hit a
// transient gateway 502 (→ no navigation, test stuck on /workflows). The
// helper re-submits on transient 5xx and waits for the /workflows/<id>
// detail navigation. A genuine create regression still fails all attempts.
await createWorkflowAndOpenDetail(page, workflowTitle);
await page.waitForLoadState("networkidle");
@ -202,45 +173,27 @@ test.describe("Workflows", () => {
/* Step 3: the debounced auto-save (800 ms) fires and the save-status
span transitions: "" → "Saving…" → "Saved".
save() (workflows/[id]/page.tsx:122-138) sets "Saving…" synchronously on
every edit, then PATCHes prompt_md and sets "Saved" (which auto-reverts to
idle after ~2 s). Under load the PATCH can transiently 502 → the catch
sets status back to "idle" so "Saved" never lands. Each keystroke re-fires
the debounced save, so we re-trigger until the PATCH succeeds.
save() (workflows/[id]/page.tsx:122-138) sets "Saving…" synchronously
on every edit, then PATCHes prompt_md and sets "Saved" (which
auto-reverts to idle after ~2 s).
REGRESSION: a removed/broken update API or save wiring shows NEITHER
"Saving…" (guard #1, save() never fires) NOR "Saved" (guard #2, PATCH
never resolves) on any attempt, so this still fails for a genuine break. */
const SAVE_ATTEMPTS = 4;
let saveConfirmed = false;
for (let attempt = 0; attempt < SAVE_ATTEMPTS && !saveConfirmed; attempt++) {
if (attempt > 0) {
// Re-fire the debounced save after a transient PATCH failure.
await page.keyboard.type(".");
}
// Guard #1: the save() handler must run (sets "Saving…" synchronously).
// PageHeader renders its actions twice — a desktop inline copy and a
// portal-mounted mobile copy — so an unscoped text locator resolves to
// two nodes and trips strict mode. Filter to the visible instance.
await expect(
page
.getByText(/^(Saving…|Saved)$/)
.filter({ visible: true })
.first(),
).toBeVisible({ timeout: 10_000 });
// Guard #2: the PATCH must resolve to "Saved" (transient 502s retried).
saveConfirmed = await page
.getByText("Saved")
"Saving…" (guard #1, save() never fires) NOR "Saved" (guard #2, the
PATCH never resolves), so this fails for a genuine break. */
// Guard #1: the save() handler must run (sets "Saving…" synchronously).
// PageHeader renders its actions twice — a desktop inline copy and a
// portal-mounted mobile copy — so an unscoped text locator resolves to
// two nodes and trips strict mode. Filter to the visible instance.
await expect(
page
.getByText(/^(Saving…|Saved)$/)
.filter({ visible: true })
.first()
.waitFor({ state: "visible", timeout: 8_000 })
.then(() => true)
.catch(() => false);
}
expect(
saveConfirmed,
"workflow prompt auto-save never reached the 'Saved' state",
).toBe(true);
.first(),
).toBeVisible({ timeout: 10_000 });
// Guard #2: the PATCH must resolve to "Saved".
await expect(
page.getByText("Saved").filter({ visible: true }).first(),
).toBeVisible({ timeout: 10_000 });
});
});
@ -281,8 +234,9 @@ test.describe("Account Settings", () => {
test("updating display name saves and persists across navigation", async ({
page,
}) => {
// This test bounds-retries its mutation + persistence steps to ride out the
// intermittent gateway 502s, so give it more headroom than the 30 s default.
// This test bounds-retries its mutation + persistence steps to converge
// past a real client-side hydration race (see below), so give it more
// headroom than the 30 s default.
test.setTimeout(120_000);
await page.goto("/account");
await expect(
@ -302,18 +256,17 @@ test.describe("Account Settings", () => {
.locator("xpath=parent::div")
.getByRole("button", { name: /save/i });
// Robustly save the new name and verify it persists. Two real hazards are folded
// into one converging retry:
// Robustly save the new name and verify it persists. This retry exists
// for a real client-side hazard, independent of infrastructure:
//
// 1) Async hydration race. The account page hydrates this input from a profile
// fetch (UserProfileContext → `if (profile?.displayName) setDisplayName(...)`).
// Under cold-start the auth state can settle late and trigger a SECOND profile
// fetch that overwrites the field AFTER we type — so the stale stored name is
// what handleSaveDisplayName persists (observed: a *previous* run's name was
// saved). We therefore (re)fill immediately before saving and re-verify the
// persisted value; if a late overwrite slipped a stale value in, the persist
// check fails and the block re-runs (auth has settled by then, so it converges).
// 2) Transient gateway 502 on the PATCH or the post-reload GET — also retried here.
// Async hydration race. The account page hydrates this input from a profile
// fetch (UserProfileContext → `if (profile?.displayName) setDisplayName(...)`).
// Under cold-start the auth state can settle late and trigger a SECOND profile
// fetch that overwrites the field AFTER we type — so the stale stored name is
// what handleSaveDisplayName persists (observed: a *previous* run's name was
// saved). We therefore (re)fill immediately before saving and re-verify the
// persisted value; if a late overwrite slipped a stale value in, the persist
// check fails and the block re-runs (auth has settled by then, so it converges).
//
// On success the label flips Save → "Saved" for ~2 s (Display-Name button only; the
// Organisation button stays "Save").
@ -321,7 +274,7 @@ test.describe("Account Settings", () => {
// REGRESSION: a broken profile PATCH / save handler never reaches "Saved" and never
// persists newName, so every attempt fails and toPass exhausts → the test fails.
await expect(async () => {
// Reload at the START of each attempt so a transient 502 on the profile GET
// Reload at the START of each attempt so a failed or stale profile GET
// (which leaves the input empty via the null-displayName fallback, with no
// client-side refetch) is retried with a fresh fetch rather than looping on a
// permanently-empty page.