feat(evals): use OpenCode 1.18.31 for GPT-6 release runs - #129
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bdff21f81f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Independent current-head verification at The frozen install passed. Fifty-three focused release, canary, and qualification tests passed. The qualification CLI rejected a canary recorded on OpenCode 1.18.6 and sealed the same campaign with a 1.18.31 canary. The packed plugin loaded in a real OpenCode 1.18.31 process without model calls. All seven GitHub checks are green, and the branch has no merge conflict. Note: GitHub Codex reviewed an earlier head. This head resolves its host-version finding. The release campaign must use the pinned 1.18.31 host; the local global |
Why
The standard release evaluator pins OpenCode 1.18.6, whose catalog does not list
openai/gpt-6-sol. A release campaign using the requested OpenAI model needs a host that resolves it and a matching packaged plugin SDK.Scope
Pin the release evaluator and
@opencode-ai/plugindependency to OpenCode 1.18.31, updatebun.lock, and useopenai/gpt-6-solwithxai/grok-4.6in the release guide. Update current-host release and canary test fixtures. Historical study contracts and retained evidence remain unchanged.Tradeoffs
The release host is part of the frozen release policy. This change requires a fresh two-provider campaign and exact-artifact canary. Earlier campaign evidence cannot be reused for the new host policy.
Blast Radius
The package's OpenCode plugin SDK and release evaluation host change together. The packed plugin loaded successfully in a model-free OpenCode 1.18.31 smoke. Model entitlement, billing, and qualification are still unmeasured.
Verification
Frozen
bun install,bun run check,bun run replay, andbun run package:smokepassed. The full check recorded 1,585 passing tests, 20 intentional skips, and no failures. Replay matched all 13 cassettes.FLOW_OPENCODE_SMOKE_VERSION=1.18.31 FLOW_LIVE_SMOKE=1 bun test tests/live-opencode-smoke.test.tspassed all 22 checks without provider inference. The pinned host catalog listsopenai/gpt-6-solas active. No paid calls ran.