Skip to content

ci: move Mac mini jobs to Namespace runners - #923

Merged
ovitrif merged 6 commits into
masterfrom
ci/namespace-mac-runner
Oct 20, 2026
Merged

ovitrif merged 6 commits into
masterfrom
ci/namespace-mac-runner

Conversation

@mdozhdev

@mdozhdev mdozhdev commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Refs: #689

This PR moves the two jobs that need the self-hosted Mac mini, iOS E2E and the Trezor emulator tests, onto Namespace macOS runners, so the Mini can be retired.

Description

  • Runs e2e-tests-local and the Trezor emulator job on namespace-profile-mac instead of [self-hosted, macOS]. Each run gets a fresh machine, so runs no longer queue behind each other or inherit leftover state.
  • Starts the regtest stack on a Namespace Linux instance that the job creates and destroys. Hosted macOS runners are VMs without nested virtualization and cannot run Docker themselves.
  • Forwards the stack's ports to the runner's 127.0.0.1 under the same numbers, so the E2E suite, the app's Env defaults and the Trezor UI tests need no changes.
  • Adds namespace-stack.sh, shared by both jobs. It creates the instance, copies the compose directory to it (bind mounts resolve on the instance), forwards ports, and tears the instance down.
  • Copies the LND TLS cert and macaroon back from the container, because the suite reads them from the local docker/lnd directory.
  • Checks Electrum with a real request rather than an open port, because a forwarded port accepts connections before electrs is ready.
  • Pins the simulator to iPhone 17 on iOS 26.2 by UDID: the runner image has several iPhone 17 devices across iOS versions. The Trezor job also pins Xcode 26.2, matching build-local.
  • Starts only the regtest services the Trezor test uses, instead of bitkit-docker's whole default stack. That stack builds lnurl-auth-server, vss-server and homegate from sources the workflow never checks out, and it only worked on the Mini because those images were already cached there.
  • Runs docker exec on the instance over nsc ssh, because exec through the Namespace Docker context returns nothing and reports success; the Trezor scripts drive the emulator through exec.
  • Maps host.docker.internal for the Trezor User Env container, which Docker Desktop provides on the Mini and plain Linux Docker does not.
  • Removes self-hosted-cleanup.yml, which only cleaned the Mini's disk.

Out of Scope

  • Org setup: the Namespace GitHub app has to be installed on synonymdev/bitkit-ios, with a macOS runner profile named namespace-profile-mac (macOS 26, Xcode 26.2 installed). Until then the changed jobs stay queued.
  • .github/workflows/e2e-tests.yml: attempts 2 and 3 still reuse the chain attempt 1 left behind, as on the Mini.
  • .github/workflows/e2e-tests.yml: build-local stays on macos-latest; only the job that needed Docker moves.
  • Deregistering the Mini runner, which needs repository admin access.
  • Superseding test: add a hosted-runner e2e path alongside the self-hosted one #689's Tailscale split, which this PR makes unnecessary.
  • BitkitUITests/TrezorBridgeDashboardUITests.swift: the two tests that fail on the new runner; the job has no green baseline on the Mini either.

Design

N/A — no UI changes.

Preview

N/A — no user-visible changes.

QA Notes

Journeys

N/A — no user-visible behaviour change.

Manual Tests

N/A

Automated Checks

  • ran the same job definitions on Namespace runners in a private copy of this repository: the regtest stack came up on an arm64 Linux instance, every forwarded port answered on 127.0.0.1 under its own number, and bitcoind RPC, Electrum and LND REST over TLS responded from the runner
  • ran the full @ios_gate suite on that setup: 14 of 14 passed on the first attempt in 56 minutes, hardware wallet included
  • ran TrezorBridgeDashboardUITests on that setup: the emulator and Bridge came up and 1 of 3 tests passed; the other two failed inside the tests (a regtest RPC helper error and an assertion). This job has no green run on master to compare with, so those two need separate investigation
  • found that docker exec through the Namespace Docker context returns no output, drops stdin and always exits 0, which is why exec now goes over nsc ssh
  • (after merge) e2e-tests-local - e2e runs on namespace-profile-mac and e2e-status reports as before
  • (after merge) dispatching ai-device-tests with trezor-emu starts the emulator on namespace-profile-mac and reaches the UI tests

🤖 Generated with Claude Code

@mdozhdev
mdozhdev force-pushed the ci/namespace-mac-runner branch from 75eb6a3 to 3ff1375 Compare October 19, 2026 14:13
@mdozhdev
mdozhdev marked this pull request as ready for review October 19, 2026 14:22
@mdozhdev

Copy link
Copy Markdown
Collaborator Author

This is now the go-to way to move iOS E2E off the Mac mini, replacing the split GitHub-hosted stack (an Ubuntu job running the regtest stack, joined to a macOS tester over Tailscale) from #689, #692 and synonymdev/bitkit-e2e-tests#208. Those PRs are back in draft.

Here the whole run is one job on a Namespace Mac: the regtest stack runs on a Namespace Linux instance and its ports are forwarded to the runner's localhost, so the test suite and the app need no changes and there is no second job to coordinate.

@greptile-apps

greptile-apps Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The changes affect two bounded CI workflows and their temporary test infrastructure.

The PR appears safe to merge, with non-blocking feedback to pin the fallback Compose download.

Findings

  1. P2 Compose changes between runs ▶

Summary

Moves the E2E and Trezor emulator jobs from the Mac mini to Namespace macOS runners.

  • The test workflows can run Docker services on a temporary Linux machine.
  • The E2E job runs on a Namespace Mac and uses a temporary regtest stack.
  • The Trezor emulator job runs on a Namespace Mac with a remote regtest stack.
  • The manually run Mini cleanup workflow is gone.

Acknowledged scope from mdozhdev: Namespace app/profile setup and Mac mini deregistration are deferred; E2E retries intentionally reuse the chain; build-local stays on macos-latest; the Tailscale split is superseded; two existing Trezor test failures are deferred because there is no green baseline on the Mini.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Namespace macOS runner] -->|Create and copy stack files| B[Namespace Linux instance]
  A -->|Docker context| B
  A -->|SSH for docker exec| B
  B --> C[Regtest and emulator containers]
  D[iOS simulator and tests] -->|Local forwarded ports| C
  C -->|Copy LND credentials| A
  A -->|Destroy after job| B
Loading

Reviews (1) · Last reviewed commit: "ci: run docker exec on the Namespace ins..." · Reviewed by Greptile

Comment thread .github/scripts/namespace-stack.sh Outdated

@talosmachina talosmachina left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 finding (1 P1). Moves iOS E2E and the Trezor emulator job off the self-hosted Mac mini onto Namespace macOS runners, with the regtest stack on an ephemeral Namespace Linux instance and its ports forwarded to 127.0.0.1. Reviewed 3ff1375, full tier.

What I checked, and 4 candidates I ruled out

Read in full: .github/scripts/namespace-stack.sh, .github/workflows/ai-device-tests.yml, .github/workflows/e2e-tests.yml, BitkitUITests/TrezorBridgeDashboardUITests.swift
Cross-repo: bitkit-e2e-tests docker/docker-compose.yml, test/helpers/constants.ts, scripts/trezor-emulator; bitkit-docker docker-compose.yml, scripts/trezor-emulator
CI: not run on this PR yet (the changed jobs are path-filtered or dispatch-only); reasoned from the code, iOS does not build on Linux.

Ruled out

  • E2E suite lacks funds on the fresh chain: bitkit-e2e-tests' ensureLocalFunds() mines its own blocks, which matches the 14/14 @ios_gate run in the description.
  • LND credentials missing locally: the new copy step pulls admin.macaroon and tls.cert to the paths constants.ts reads.
  • Ports the app or tests dial but are not forwarded: Electrum 60001, bitcoind RPC 43782, LND 8080/9735/10009, backup 3003 and Bridge 21325 are all in the forward lists; the Trezor job's trezor-emulator script uses 9001 and 23902, both forwarded.
  • docker exec silently succeeding through the Namespace context: the wrapper routes exec over nsc ssh, and GITHUB_PATH puts it first for later steps.
  • Cold reader: the finding stood.

Merge confidence: 3/5, one P1 in the Trezor job.

Comment thread .github/workflows/ai-device-tests.yml

@jvsena42 jvsena42 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code scan: no HIGH/MEDIUM. One LOW is inline.

Checked and clean:

  • Secrets: e2e-tests-local and trezor-emu take no repo secrets. Signing and release workflows don't use the Mini.
  • Gating: e2e-status still requires e2e-tests-local == success, so a missing Namespace profile leaves the check pending instead of passing.
  • Cleanup: teardown runs under always(), and --duration 4h bounds orphaned instances.
  • Routing: the test suite and both Trezor scripts use plain docker exec, which the wrapper routes over nsc ssh.

Worth checking, not a finding: fork PRs run namespace-stack.sh from the merge ref on a runner holding nsc workspace credentials. The Mini had the same exposure, so this isn't worse, but check the Namespace profile's fork policy, or gate the job on head.repo.full_name == github.repository.

Not approving yet: e2e-tests-local, the job this PR moves, is still running at a94a902. I'll approve once it's green. The red Run Tests is a simulator launch denial (The request was denied by service delegate (SBMainWorkspace)) in a workflow this PR doesn't touch; it needs a re-run before merge.

Device gate: no device check needed (CI only); perf: not relevant; l10n: no UI text changed.

Comment thread .github/scripts/namespace-stack.sh
@mdozhdev

Copy link
Copy Markdown
Collaborator Author

@lnurl_1 fails on the Namespace runner. It looks like an app issue, not the runner setup.

Results on this PR: 13 of 14 @ios_gate tests pass, hardware wallet included. @lnurl_1 failed all 6 attempts across two runs (39272354166, 39289000849), always at the same step.

What happens:

  • The first pay request (amount entered, with a comment) completes: SendSuccess shows and the activity checks pass. So LND, the channel, and the spec's own LNURL server on localhost:30001 all work through the runner.
  • The second pay request has minSendable == maxSendable (222 sats, no comment), so the app should go straight to review. The send sheet opens but stays empty, and ReviewAmount-primary never appears (lnurl.e2e.ts:235). The failure screenshot and video are in the runs' e2e-artifacts_e2e_* artifacts.

What differs from the Mini:

  • The Mini runs the iOS 26.0.1 simulator. The Namespace image starts at iOS 26.2 and has no 26.0.1 runtime.
  • The same spec passed on Namespace with iOS 26.2 on 2026-10-15, with the app built from 2026-10-12 master. About 190 commits landed since, several of them changing how payment links and send sheets open.

So the likely cause is a recent app change that breaks the fixed-amount LNURL pay flow on iOS 26.2+, which current phones run and the Mini's older simulator does not exercise. That is an inference from these results, not a confirmed root cause. It needs someone who knows the send flow; it is outside this PR.

@mdozhdev
mdozhdev requested a review from jvsena42 October 20, 2026 09:56
@mdozhdev

Copy link
Copy Markdown
Collaborator Author

Update: E2E is green on this PR. 39404241526 on 28a9e27 passed e2e-tests-local and e2e-status on the Namespace runner.

Correction to my LNURL comment above: @lnurl_1 is intermittent, not a consistent failure. In this run attempt 1 passed 13 of 14 (@lnurl_1 failed at the same empty send sheet), and attempt 2 passed @lnurl_1. Attempt 2 runs on the chain attempt 1 left behind, so it is not an identical rerun, but it shows the fixed-amount pay flow can complete on iOS 26.2. The empty sheet looks like a timing race in the app rather than a flow that is broken on 26.2; it is still worth a look, outside this PR.

Current checks:

Since the last review: the detect-changes gap is fixed (28a9e27), Talos's funding finding is fixed (a94a902), and the Compose download is pinned (93cb30a).

@jvsena42 jvsena42 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. My LOW is fixed in 28a9e27, and e2e-tests-local is green on the Namespace runner at this head (run 39404241526), so the condition I held the approval on is met.

  • @lnurl_1 failed once and passed on retry within that run: a fixed-amount LNURL pay opened an empty send sheet on the iOS 26.2 simulator. The first LNURL pay in the same spec succeeds through the forwarded ports, so this looks like an app timing race rather than the runner move.
  • The red SwiftFormat check is on master's ActivityRow.swift:104, the same on the last three master heads; #1040 fixes it.

Device gate: no device check needed (CI only); perf: not relevant; l10n: no UI text changed.

@ovitrif

ovitrif commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

@mdozhdev the red SwiftFormat check here comes from master's own files, which #1040 fixed (merged). Syncing this branch with master turns it green; if it still fails after that, run SwiftFormat 0.64.0 on this PR's changed files.

@jvsena42

Copy link
Copy Markdown
Member

@mdozhdev please, rebase this branch to get the lint fix

mdozhdev and others added 6 commits October 20, 2026 20:11
The self-hosted Mac mini ran Docker and the iOS Simulator on one machine.
Hosted macOS runners are VMs without nested virtualization, so the regtest
stack now runs on a Namespace Linux instance created per job, with its
ports forwarded to the runner's loopback under the same numbers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…re starting the emulator

The hardware wallet specs start their own Trezor emulator and reach its bridge
on 127.0.0.1:21325. The LNURL spec runs its own server on 127.0.0.1:30001, so
that port must stay free on the runner. trezor-emulator start checks the bridge
before returning, so ports are forwarded first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
docker exec through the attached context returns no output, drops piped
stdin and always exits 0, so the Trezor emulator scripts, which drive the
emulator through docker exec, silently did nothing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each run starts a new chain, and bitcoinsetup mines a single block whose
coinbase is immature, so sendtoaddress in the Trezor UI test failed with
insufficient funds. The Mini kept a funded wallet because its teardown
never removed the bitcoin_home volume.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@mdozhdev
mdozhdev force-pushed the ci/namespace-mac-runner branch from 28a9e27 to 9346e43 Compare October 20, 2026 18:24
@ovitrif
ovitrif merged commit 06ccb1d into master Oct 20, 2026
18 checks passed
@ovitrif

ovitrif commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

Merged, as it was approved and green.

@ovitrif
ovitrif deleted the ci/namespace-mac-runner branch October 20, 2026 20:01
@ovitrif ovitrif added this to the 3.1.0 milestone Oct 20, 2026
@ovitrif

ovitrif commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

Triage: milestone 3.1.0. A CI change that moves the Mac mini jobs to Namespace runners and ships with the next minor; it closes no issue, so this PR carries the milestone itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants