mirror of
https://github.com/librespeed/speedtest.git
synced 2026-08-04 06:39:45 +00:00
chore/playwright mode test plan (#778)
* ci: make standalone playwright manual and document e2e gating * docs(test): refine playwright plan with implemented CI strategy --------- Co-authored-by: Stefan Stidl <stefan.stidl@ffg.at>
This commit is contained in:
parent
49f54a5c3c
commit
3bd9e1b7af
3 changed files with 21 additions and 10 deletions
2
.github/workflows/docker-publish.yml
vendored
2
.github/workflows/docker-publish.yml
vendored
|
|
@ -24,6 +24,8 @@ env:
|
|||
IMAGE_NAME: ${{ github.repository }}
|
||||
|
||||
jobs:
|
||||
# Keep e2e in this workflow so Docker image build/push is hard-gated by test success.
|
||||
# Do not move automatic e2e back to playwright.yml, otherwise master pushes run duplicate e2e jobs.
|
||||
e2e:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
|
|
|
|||
6
.github/workflows/playwright.yml
vendored
6
.github/workflows/playwright.yml
vendored
|
|
@ -1,9 +1,9 @@
|
|||
name: Playwright E2E
|
||||
|
||||
# Manual-only by design:
|
||||
# - Automatic e2e runs that gate Docker image builds live in docker-publish.yml.
|
||||
# - Keeping this workflow manual avoids running the same e2e suite twice on master pushes.
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
|
|
|
|||
|
|
@ -3,6 +3,12 @@
|
|||
## Objective
|
||||
Build a deterministic Playwright test suite that validates LibreSpeed behavior across all supported deployment modes and UI design modes, without asserting real network throughput values.
|
||||
|
||||
## Current Status
|
||||
- Phase 1 is implemented and passing in Chromium.
|
||||
- Added regression coverage for the classic standalone "No servers available" issue path.
|
||||
- Docker image workflow is hard-gated by e2e (`build` depends on `e2e`).
|
||||
- Standalone Playwright workflow is manual-only to avoid duplicate e2e runs.
|
||||
|
||||
## Modes to Cover
|
||||
|
||||
### Docker runtime modes
|
||||
|
|
@ -68,11 +74,12 @@ Expectations:
|
|||
## Playwright Architecture
|
||||
|
||||
### Files
|
||||
- `playwright.config.ts`
|
||||
- `tests/e2e/modes.spec.ts` (runtime-mode smoke)
|
||||
- `tests/e2e/design-switch.spec.ts` (classic/modern/switch overrides)
|
||||
- `tests/e2e/helpers/env.ts` (base URLs + mode metadata)
|
||||
- `tests/e2e/helpers/ui.ts` (shared selectors, start/abort helpers)
|
||||
- `playwright.config.js`
|
||||
- `tests/e2e/modes.spec.js` (runtime-mode smoke)
|
||||
- `tests/e2e/design-switch.spec.js` (classic/modern/switch overrides)
|
||||
- `tests/e2e/classic-standalone-regression.spec.js` (revert regression guard)
|
||||
- `tests/e2e/helpers/env.js` (base URLs + mode metadata)
|
||||
- `tests/e2e/helpers/ui.js` (shared selectors, start/abort helpers)
|
||||
|
||||
### Environment boot
|
||||
- `docker compose -f tests/docker-compose-playwright.yml up -d --build`
|
||||
|
|
@ -108,8 +115,9 @@ Use role/text selectors anchored on stable labels and IDs already in pages; avoi
|
|||
- Mitigation: maintain per-design helper selectors with minimal coupling.
|
||||
|
||||
## CI Proposal
|
||||
- Trigger on PR + main branch
|
||||
- Build test image once, run mode services in parallel ports
|
||||
- Docker workflow runs e2e first, then build/push only if e2e passes
|
||||
- Standalone Playwright workflow is `workflow_dispatch` only for manual branch runs
|
||||
- Keep a single automatic e2e path to avoid duplicate runs
|
||||
- Playwright retries: `1` in CI, `0` locally
|
||||
- Upload traces/screenshots on failure only
|
||||
- Browser scope for v1: Chromium only
|
||||
|
|
@ -118,3 +126,4 @@ Use role/text selectors anchored on stable labels and IDs already in pages; avoi
|
|||
1. Browser scope for v1: Chromium only.
|
||||
2. Telemetry checks are deferred to Phase 3.
|
||||
3. `backend` mode tests assert backend endpoint contracts only.
|
||||
4. Automatic e2e gating lives in Docker workflow; standalone Playwright workflow is manual.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue