Feat/smoke verdict - #9
Merged
Merged
Conversation
Stop-SmokeApp's macOS quit left a stale $LASTEXITCODE from pkill (exit 1 when the process was already gone), which GitHub's pwsh step wrapper turns into a failed step; every macOS legacy pass was failing its own launcher step. Reset $LASTEXITCODE after Stop-SmokeApp and exit 0 after a recorded run, since the verdict step is what should judge pass/fail, not a leftover native tool exit code. Also bound the macOS quit with an AppleScript timeout and a SIGKILL fallback so a hung app cannot stall the step, clear stale app-*.log/result.json/run.json and the extraction work directory at the start of a run so a retry in the same job cannot read a prior run's leftovers, and pipe unzip/xattr/chmod output to Out-Null in Find-SmokeTarget so it cannot leak into the function's return value. Extract the post-loop mode/timedOut/legacyAlive/exitCode decision into a pure Get-SmokeRunOutcome function with its own Pester coverage (exited-before-start, legacy-alive, start-line-seen-late, verdict-mode-timeout, and per-platform exit code handling), and switch the existing tests in this file to TestDrive: for temp paths.
Run the verdict step even when the launch step failed (if: !cancelled()), and when run.json itself is missing, report FAILED with an explicit reason (the launcher never got as far as recording a run) instead of silently defaulting to verdict mode with null fields. Print the last 50 lines of app-stdout.log and app-stderr.log inside ::group:: blocks whenever the verdict fails, so a failed run's output is visible in the job log without opening artifacts. Move Set-StrictMode from file scope to the top of each function in SmokeVerdict.ps1: dot-sourcing the file (Hermes CI does) was turning strict mode on in the caller's own scope. Document the run.json contract (mode, exitCode, timedOut, legacyAlive) on the smoke-verdict action's output_dir input and in Invoke-SmokeVerdict.ps1's header comment. Save and restore $env:GITHUB_OUTPUT / $env:GITHUB_STEP_SUMMARY around each test instead of removing them, since a real runner already has both set, and switch temp directories to TestDrive:.
Add a launcher-e2e job (ubuntu-latest, macos-latest) that runs the real ./actions/smoke-test composite action, so GitHub's own pwsh step wrapper applies, against three fixtures per OS with their own releases_dir/output_dir: a legacy app that never speaks the smoke protocol, and smoke-aware apps that report PASSED and FAILED. Asserts each fixture's result output in a final step that fails the job on any mismatch. Add -SkipPublisherCheck to the Install-Module Pester fallback, since the in-box Pester 3.4 on windows-latest is signed by a publisher not already trusted on the runner and otherwise blocks the install.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.