Skip to content

test(e2e): run a suite against a local PlaceOS stack #326

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MrYuion merged 3 commits into from
Oct 21, 2026
Merged

test(e2e): run a suite against a local PlaceOS stack #326

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c684e3f
fix(auth): use the root redirect_uri on any localhost dev server
MrYuion Oct 21, 2026
4d6c311
test(e2e): run a suite against a local PlaceOS stack
MrYuion Oct 21, 2026
bc3bd3c
ci(e2e): run the local stack suite on a self-hosted runner
MrYuion Oct 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
201 changes: 201 additions & 0 deletions .github/workflows/e2e-local.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
name: E2E LOCAL (advisory)

# Runs the local-backend e2e suite (e2e/src/local) against the isolated
# PlaceOS stack in e2e/stack, on a self-hosted runner. See "CI" in
# e2e/README.md.
#
# ADVISORY: do not add this job to branch protection or the required checks
# until the suite has a track record (no false failures, a known flake rate).
# That is a repository setting; nothing in this file can make it required.
#
# There is NO `pull_request` trigger, on purpose. This repository is public and
# the runner is self-hosted, so a fork pull request could run any code on a
# machine in an internal network. Every trigger here needs write access to the
# repository. Do not add `pull_request` without a move to `ubuntu-latest`.
on:
workflow_dispatch:
push:
branches:
# The branches BUILD deploys from
- develop
- 'release/**'
- 'rc/**'
# To work on this suite and this workflow
- 'e2e/**'
schedule:
# Daily. Scheduled runs use the default branch only. 30 minutes after
# the user-interfaces nightly, which can share the runner.
- cron: '40 1 * * *'

# The event is in the group, so a push to develop does not cancel a nightly
# run on the same ref.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref }}
cancel-in-progress: true

# The job only reads the code. The runner machine is persistent, so it must not
# hold a token that can push.
permissions:
contents: read

jobs:
e2e-local:
name: 'local e2e (advisory, does not block builds)'
# Targeted by its custom label, so the machine can change without an
# edit here. The runner must be registered for this repository.
runs-on: [self-hosted, placeos-e2e]
timeout-minutes: 30
env:
E2E_BACKEND_URL: https://localhost:9543
E2E_APP_URL: http://localhost:4216
TZ: 'Etc/UTC'
steps:
# Checkout otherwise writes the token to .git/config, which stays on
# this machine
- uses: actions/checkout@v6
with:
persist-credentials: false

# No actions/cache and no setup-node: the workspace and toolchain
# stay on this machine, and an upload to the remote cache costs more
# than it saves. no-cache keeps bun out of the GitHub cache. Keep the
# version the same as test.yml and build.yml.
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: '1.3.11'
no-cache: true

- name: Install dependencies
run: bun install --frozen-lockfile

- name: Setup version
run: bun run postinstall

# This machine is not a fresh VM. An aborted run can leave the
# stack up or the dev server port held.
- name: Reclaim the machine from an earlier run
run: |
e2e/stack/down.sh --volumes || true
lsof -nP -iTCP:4216 -sTCP:LISTEN -t 2>/dev/null | xargs kill -9 2>/dev/null || true
docker image prune -f || true

- name: Install Playwright browser
run: bunx playwright install chromium

# Without a pull, `latest` stays the image of the day the runner was
# set up. A pull that fails or stalls must not fail the run: the
# stack then starts on the images that are present, and the next
# step records which.
- name: Pull the current backend images
timeout-minutes: 15
continue-on-error: true
run: e2e/stack/up.sh --pull-only

# Needs GitHub (frontend-loader clones PlaceOS/www-core for the
# /login page) and Docker Hub.
- name: Bring up the local PlaceOS stack
timeout-minutes: 10
run: e2e/stack/up.sh

# The PlaceOS images track ${PLACEOS_TAG:-latest}, so a result can
# change with no frontend commit. Record the image ids, so a changed
# nightly can be explained. Set PLACEOS_TAG in `env` to pin them.
- name: Record backend inputs
run: |
{
echo '### Backend inputs'
echo
echo '| image | id |'
echo '|---|---|'
docker compose -p backoffice-e2e -f e2e/stack/docker-compose.yml images \
--format json 2>/dev/null \
| node -e '
let raw = "";
process.stdin.on("data", (d) => (raw += d));
process.stdin.on("end", () => {
let rows = [];
try { rows = JSON.parse(raw); } catch { return; }
for (const r of rows) {
console.log(`| ${r.Repository}:${r.Tag} | \`${(r.ID || "").slice(0, 19)}\` |`);
}
});
' || true
echo
echo "www-core HEAD: \`$(git ls-remote https://github.com/PlaceOS/www-core HEAD | cut -f1)\`"
} >> "$GITHUB_STEP_SUMMARY"

- name: Run local e2e
run: bun run e2e:local

- name: Summarise results
if: ${{ always() }}
run: node e2e/stack/ci-summary.mjs >> "$GITHUB_STEP_SUMMARY"

# Every service, not a selected list: the log that explains a
# failure is often from a service nobody expected.
- name: Collect backend logs
if: ${{ failure() }}
working-directory: e2e/stack
run: |
mkdir -p ../../e2e-logs
docker compose -p backoffice-e2e ps --all > ../../e2e-logs/_ps.txt 2>&1 || true
for svc in $(docker compose -p backoffice-e2e config --services 2>/dev/null); do
docker compose -p backoffice-e2e logs --no-color "$svc" \
> "../../e2e-logs/$svc.log" 2>&1 || true
done

- name: Upload Playwright report
if: ${{ always() }}
uses: actions/upload-artifact@v4
with:
name: playwright-report-local
path: |
dist/.playwright/e2e-local/report
dist/.playwright/e2e-local/results.json
retention-days: 14
if-no-files-found: warn

- name: Upload failure artifacts
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: playwright-failures-local
path: |
dist/.playwright/e2e-local/test-output
e2e-logs
retention-days: 14
if-no-files-found: warn

# The volumes go too: every run starts from a cold stack
- name: Tear down the stack
if: ${{ always() }}
run: e2e/stack/down.sh --volumes

# Only on failure, so a green run stays quiet. Plain curl, not a
# third-party action at a mutable ref: this runs on a persistent
# machine. The secrets go through `env`, so they are not on the
# command line.
- name: Notify on failure
if: ${{ failure() }}
continue-on-error: true
env:
STATUS_URL: ${{ secrets.STATUS_URL }}
CHAT_URL: ${{ secrets.CHAT_URL }}
REPO: ${{ github.repository }}
COMMIT: ${{ github.sha }}
BRANCH: ${{ github.ref_name }}
RUN_ID: ${{ github.run_id }}
run: |
node -e '
process.stdout.write(JSON.stringify({
chat_url: process.env.CHAT_URL,
name: process.env.REPO,
commit: process.env.COMMIT,
branch: process.env.BRANCH,
url: `https://github.com/${process.env.REPO}/actions/runs/${process.env.RUN_ID}`,
pipeline_name: "backoffice e2e local (advisory) FAILED",
status: "failure",
}));
' | curl --silent --show-error --fail --max-time 30 \
--header 'Content-Type: application/json' --data @- "$STATUS_URL"
4 changes: 4 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,7 @@ vitest.config.*.timestamp*
migrations.json
src/env/version.ts
.nx/migrate-runs

# E2E auth artifacts: bearer tokens. Never commit.
e2e/.auth
e2e/.env
5 changes: 4 additions & 1 deletion CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@ bun run build # Production build
bun run test # Unit tests (Vitest)
bun run test:ci # Tests sequentially (CI mode)
bunx playwright install --with-deps
bun run e2e # Playwright E2E tests
bun run e2e # Playwright E2E tests (mock backend)
e2e/stack/up.sh # Local PlaceOS stack in Docker (https://localhost:9543)
bun run e2e:local # Playwright E2E tests against the local stack
e2e/stack/down.sh # Stop the stack (--volumes to remove its data)

# Linting
bun run lint # ESLint
Expand Down
8 changes: 8 additions & 0 deletions e2e/.env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,11 @@ USE_MOCK=true
# Test user credentials for live environment
# TEST_USER_EMAIL=admin@example.com
# TEST_USER_PASSWORD=password

# Local backend suite (`bun run e2e:local`). Both URLs must be on a loopback
# host: the suite refuses any other host.
# E2E_BACKEND_URL=https://localhost:9543
# E2E_APP_URL=http://localhost:4216
# E2E_ADMIN_EMAIL=support@place.tech
# E2E_ADMIN_PASSWORD=development
# E2E_STAFF_PASSWORD=e2e-staff-development
148 changes: 144 additions & 4 deletions e2e/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
# E2E Tests for PlaceOS Backoffice

These Playwright tests run the app against the mock backend in
There are two suites:

- **Mock** (`src/*.spec.ts`): the app runs against the mock backend. CI runs
this suite. Most of this document is about it.
- **Local backend** (`src/local/*.spec.ts`): the app runs against a real
PlaceOS stack in Docker. See [Local backend](#local-backend).

The mock tests run the app against the mock backend in
`src/app/mocks/`. Each test gets a new browser context, so it starts with
fresh mock data. The mock backend keeps creates, edits and deletes in session
storage, so a change stays after `page.reload()` in the same test.
Expand All @@ -26,9 +33,9 @@ The config starts the dev server, or uses the server that runs on port 4200.
CI runs `chromium` and `mobile-chrome` for pull requests to `develop`, and
all projects on other branches.

`USE_MOCK=false` stops the tests from switching on the mock backend. There is
no sign-in flow for a live server, and most tests expect the mock data, so
use this only to debug.
`USE_MOCK=false` stops the tests from switching on the mock backend. Most
tests expect the mock data, so use this only to debug. To test against a real
backend, use the [local backend](#local-backend) suite.

## Coverage

Expand Down Expand Up @@ -91,3 +98,136 @@ Rules:
assertions (`toHaveText`, `toHaveURL`, `toHaveCount`) wait for the app.
6. The item list is virtual and shows only part of a long list. Search for an
item before you look for it in a long list.

## Local backend

These specs run against a PlaceOS stack on your computer. They make, change
and delete real data, so the config refuses a backend that is not on a
loopback host. The stack and the suite are adapted from the
`PlaceOS/user-interfaces` e2e stack.

You need Docker with about 2 GB of free memory for the stack.

```bash
e2e/stack/up.sh # start the stack (--fresh: remove its data first, --pull: pull new images)
bun run e2e:local # run the specs
bun run e2e:local -- zones.spec # run one spec file
bun run e2e:local -- --ui # UI mode
e2e/stack/down.sh # stop the stack (--volumes: remove its data)
```

### The stack

`e2e/stack/` is its own compose project (`backoffice-e2e`) with its own
network, volumes and secrets. It uses ports 9543 (HTTPS) and 9180 (HTTP). Thus
it can run at the same time as PlaceOS/local (8443) and the user-interfaces
e2e stack (9443).

It has: postgres, elasticsearch, redis, search-ingest, frontend-loader, auth,
rest-api, staff-api, nginx and init. It does not have core, so no driver
compiles and no module starts. Add core from the user-interfaces stack when a
spec needs it.

`up.sh` makes the secrets in `e2e/stack/.secrets/` (not in git), starts the
services, waits for the API and the login page, and runs `init`. `init` makes
the authority, the admin user (`support@place.tech` / `development`) and a
zone tree.

`frontend-loader` is necessary. It copies `PlaceOS/www-core` into the `www`
volume, and the platform `/login` page comes from there. Thus the first
start needs access to GitHub.

### How a run works

1. Playwright starts the dev server on port 4216, with its proxy
(`config/proxy.conf.js`) pointed at the stack. It does not use port 4200,
because a dev server there can proxy to a shared deployment.
2. The `setup` project (`support/stack.setup.ts`) checks that the stack
answers, then seeds it (`support/seed.ts`). The seed is idempotent:
- It registers an OAuth app for `http://localhost:4216/oauth-resp.html`.
ts-client uses `Md5(redirect_uri)` as its client id, and the sign-in
fails if no app has that id.
- It makes the authority's `login_url` absolute. A new authority has a
relative `login_url`, and ts-client then sends the browser to
`http://localhost/login`, which has no port.
- It makes a user that is not an admin (`e2e-staff@place.tech`).
3. The setup signs in as each user through the API (OAuth code + PKCE) and
writes a storageState to `e2e/.auth/` (not in git). These files hold
bearer tokens.
4. The `local` project runs `src/local/*.spec.ts`. `page` is signed in as
the admin.

### CI

`.github/workflows/e2e-local.yml` runs this suite on a self-hosted runner
with the label `placeos-e2e`. It runs:

- on a push to `develop`, `release/**`, `rc/**` and `e2e/**`
- every day at 01:40 UTC
- from **Run workflow**

It has no `pull_request` trigger. This repository is public, so a fork pull
request could run any code on the runner. Do not add the trigger, except if
the job moves to `ubuntu-latest`.

The job is advisory. Do not make it a required check until it has a record of
runs with no false failures.

Each run removes what an earlier run left, pulls the images, starts the stack
with no data, runs the suite and removes the stack and its volumes. The step
summary shows the image ids and the flaky specs. On failure, the artifacts
have the traces, the videos and the log of each service.

To set up the runner machine, follow
[`SELF_HOSTED_RUNNER.md`](https://github.com/PlaceOS/user-interfaces/blob/develop/e2e/stack/SELF_HOSTED_RUNNER.md)
in `PlaceOS/user-interfaces`, with these changes:

- Register the runner for `PlaceOS/backoffice` (a repository runner serves
one repository), or use an organisation runner for both repositories.
- The suite uses ports 9543, 9180 and 4216, and the user-interfaces suite uses
9443, 9080 and 4214. Thus the two jobs can run on one machine at the same
time if it has the memory.

### Write a local spec

```typescript
import { ENGINE_API, uniqueName } from '../../support/api';
import { expect, test } from '../../support/fixtures';
import { FeaturePage } from '../pages';

test('AC-ZON-004-5: a renamed zone is saved', async ({ page, api }) => {
const res = await api.post(`${ENGINE_API}/zones`, {
data: { name: uniqueName('E2E Zone') },
});
const zone = await res.json();
try {
// ...change the zone in the UI with FeaturePage...
// ...then check the result through `api`, not only on the page
} finally {
await api.delete(`${ENGINE_API}/zones/${zone.id}`);
}
});
```

Rules:

1. Import `test` and `expect` from `support/fixtures`. The fixtures are:
`page` (signed in as the admin), `staffPage` (signed in as a user that is
not an admin) and `api` (an API context with an admin bearer).
2. Each spec makes its own data with `uniqueName()` and removes it in a
`finally`. Specs run in parallel on the same stack.
3. Check the result through the API. The page can show a change that the
backend did not save.
4. Check access rules as `staffPage`. A sys_admin passes every check, so the
same test as the admin proves nothing.
5. A spec about sign-in starts with no token:
`test.use({ storageState: { cookies: [], origins: [] } })`. See
`src/local/login.spec.ts`.
6. Make the new assertion fail one time, and make sure that it fails for the
correct reason.

| Spec | Covers |
| ---------------------- | ---------------------------------------------------- |
| `local/login.spec.ts` | Sign-in through the platform login page |
| `local/access.spec.ts` | An admin opens domains; other users get the 403 page |
| `local/zones.spec.ts` | Create, rename and delete a zone, checked on the API |
Loading
Loading