0008 — End-to-end tests with Playwright + CI integration
Status: Accepted Date: 2026-11-04 Amended by: ADR-0039 (the app supports mobile viewports, and the suite covers them in emulation) and ADR-0057 (CI runs the suite in duration-balanced shards)
Context and Problem Statement
Section titled “Context and Problem Statement”The web frontend shipped through v0.2 on manual verification alone.
Every reviewer had to boot cargo run -- --serve, then a Vite dev
server, then click through the OAuth flow, then load a symbol, then
click through the indicator picker — and any regression in OAuth
redirect handling, chart rendering, or the modal interaction was
caught only if a reviewer happened to exercise that exact path.
v0.1’s “scaffold” did run HTTP-surface checks against chartgen, but
those don’t cover the browser — the real contract the frontend has
to honor is the one an actual browser sees (PKCE redirects, SPA
routing after ?code=&state= scrubbing, klinecharts canvas render,
Kobalte Dialog portal behavior, localStorage persistence). None of
those are testable from curl.
We need an automated end-to-end suite that:
- Runs in CI on every PR, against the real chartgen binary.
- Covers the OAuth 2.1 PKCE flow all the way through.
- Asserts the klinecharts canvas is actually rendered, not just that the page painted.
- Covers the indicator-picker modal add/remove/persist loop described in ADR-0005.
- Keeps the PR surface small enough that future engineers will actually maintain it.
Decision Drivers
Section titled “Decision Drivers”- Reproducibility.
npm run test:e2efromweb/must be the only local command; the suite launches its own backend and frontend. - CI-integrable. Existing GitHub Actions runners, no external services, no tunneling localhost to a cloud browser farm.
- TypeScript-native. The frontend is TS already; the suite lives next to the code it tests so the same people who write the app can maintain it.
- Low-cost matrix for v0.2. Chromium-only is enough for “does the OAuth flow still work”; cross-browser coverage is a later tradeoff.
- Full control. No cloud service fees, no account lock-in, no data egress surprises.
Considered Options
Section titled “Considered Options”Selenium / WebDriverIO. Mature, but Playwright’s auto-waiting and first-class TypeScript definitions are a meaningfully better fit for a single-engineer team. Selenium in particular carries a lot of setup-friction tax (driver binaries, Grid, Java bridges) for a use-case where one process per spec is plenty. WebDriverIO is a reasonable alternative but its ecosystem is smaller than Playwright’s in 2026 and its webServer-style fixture story is not built-in.
Cypress. The iframe-based architecture makes multi-tab and
multi-origin flows awkward — and the OAuth redirect here technically
round-trips via /authorize on a separate port (9315). Cypress’s
same-origin assumption has historically fought that pattern. The
tool also spent years Chrome-only; cross-browser is possible now but
not a reason to prefer it over Playwright.
Cloud browser service (Browserbase, BrowserStack, LambdaTest).
None of these can reach localhost:9315 without a tunnel. Tunneling
in CI is overhead without obvious benefit; we’d still want to
self-host the chartgen process for reproducibility, at which point
the cloud browser is just latency.
Manual-only, status quo. The thing we’re replacing. Rejected on principle — unreviewable regressions are piling up and every PR reviewer pays the cost.
Decision Outcome
Section titled “Decision Outcome”Chosen: Playwright, Chromium only, webServer-managed fixtures.
- Pure TypeScript logic is covered separately by Vitest (
npm test,src/**/*.test.ts, node environment); see the Testing guide. - Suite lives at
web/tests/e2e/. Spec files:smoke.spec.ts(cold-boot happy path) andindicator-picker.spec.ts(ADR-0005 modal flow). A shared fixture infixtures/app.tshandles the cold-boot + OAuth + initial Load so specs can opt in. web/playwright.config.tsdeclares twowebServerentries:cargo run -- --serve --port $CHARTGEN_PORTin the repo root (withCHARTGEN_DATA_SOURCE=samplefor deterministic offline bars) andnpm run dev -- --port $WEB_PORT --strictPortinweb/.CHARTGEN_PORT(default 9315) andWEB_PORT(default 5173) are configurable so parallel checkouts do not collide; the Vite proxy target follows viaCHARTGEN_DEV_TARGET. Servers already listening on those ports are reused only whenPW_REUSE_SERVER=1is set (never on CI); otherwise Playwright fails fast on a busy port, so a leftover server from another checkout is never tested by mistake. Reuse is an opt-in for fast local loops against servers you started from this checkout. Chartgen gets a 300-second timeout because a cold build is measured in minutes on CI without a cache; on a warm Swatinem cache it’s seconds. A debug build is used because it compiles faster than release and is fast enough here.fullyParallel: falseandworkers: 1: chartgen’s OAuth store is process-global, and two tests racing through PKCE will collide on state.- Single Chromium project — Firefox/WebKit are deferred until there’s a concrete regression they’d catch. Desktop viewport only; mobile isn’t supported by the app in v0.2 anyway.
- CI job
e2ein.github/workflows/ci.ymlgates onbuild-release, caches~/.cache/ms-playwright, runsnpm run test:e2e, and uploadsplaywright-report/+test-results/as artifacts on failure.
Not in scope for this ADR (explicitly deferred):
- Visual regression snapshots (pixel diffs on chart rendering).
- SSE stream content assertions beyond “connection opens”.
- Multi-browser matrix.
- Mobile viewports.
- Axe-core accessibility audit.
- Order / alert flows (not in the v0.2 frontend).
Consequences
Section titled “Consequences”Positive. CI now catches frontend regressions before merge: OAuth
flow breakage, chart render failure, picker modal layout bugs, and
localStorage persistence drift. The specs double as executable
documentation of the OAuth PKCE flow — a new contributor can read
smoke.spec.ts and see what the happy path looks like. The real
chartgen server runs unchanged under test.
Negative. CI wall-clock grows by roughly 2-3 minutes (Chromium
install + build + two specs). The test retries once on first
run to smooth OAuth redirect timing, which hides first-attempt
flakiness; we accept that in exchange for lower noise. Every future
indicator added to the default seed has to be consistent with the
smoke test’s expectations about .pane-header count and RSI
presence.
Neutral. web/package.json gains @playwright/test and vitest devDeps and
two scripts (test, test:e2e). The suite is skipped
entirely in environments without a browser runtime — CI provides it
via playwright install --with-deps chromium; local developers who
skip that step get a clear error from Playwright.