From 0b07dade333cc2917f1926a397224acfcfe1e040 Mon Sep 17 00:00:00 2001 From: Jacob Prall Date: Wed, 28 Oct 2026 13:09:55 -0700 Subject: [PATCH] Polish docs for first deploy and adding repositories; fail fast on rejected egress - README: one-line intro, Get running moved up with the Quickstart linked, a no-UI note, an Example answer heading, and Built on Render (#41, #44). - Quickstart: where to copy API_TOKEN, a /healthz check, scoped API key permissions per service, a prefilled GitHub token link, and an Add a repository section with the cron rollout order (#41, #42, #43, #44). - snapshot_refresh logs the build commit and repository names (#43). - A 400/422 allow-list rejection is SandboxMisconfiguredError and returns the misconfigured reply instead of retrying as an infrastructure error (#1). Co-authored-by: Cursor --- README.md | 55 ++++++++++++++++-------------- docs/0_QUICKSTART.md | 62 ++++++++++++++++++++++++++++++---- docs/1_EXTENSION.md | 4 ++- docs/2_ARCHITECTURE.md | 3 +- render.yaml | 1 + src/cron.ts | 19 +++++++---- src/sandbox/index.ts | 2 +- src/sandbox/interface.ts | 5 +++ src/sandbox/render/sandbox.ts | 3 +- src/sandbox/render/snapshot.ts | 4 +-- src/tasks.ts | 6 +++- test/cron.test.ts | 8 +++-- test/sandbox-render.test.ts | 7 +++- test/tasks.test.ts | 11 +++++- 14 files changed, 140 insertions(+), 50 deletions(-) diff --git a/README.md b/README.md index 953a31b..296ff41 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,39 @@ # codebase-researcher -A template for getting evidence-backed answers to questions about how your code behaves. Post a question to an HTTP API, or run `pnpm ask "..."`. An AI coding agent reads your repositories in a fresh, locked-down Render Sandbox and answers with file-and-line citations that are checked against the exact commit it read. +Ask how your code behaves and get an answer with file-and-line citations, checked against the exact commit an AI agent read in a locked-down Render Sandbox. [![Deploy to Render](https://render.com/images/deploy-to-render-button.svg)](https://render.com/deploy?repo=https://github.com/render-examples/codebase-researcher) The button deploys this repository as is, investigating its example corpus: three public repositories (the Render CLI, SDK, and agent skills). To point it at your own code, fork first and deploy your fork. -Every answer has the same structure: a short answer, a confidence level, what was observed in code, what was inferred, what was not established, file-and-line evidence, a suggested owner, and an escalation-ready question for engineering. When the question is ambiguous, the agent asks one clarifying question instead, and you answer it as a follow-up. A completed poll returns the answer under `reply.result`: +The researcher is an HTTP API with no UI. You ask questions with `pnpm ask` or `curl`, so opening the service URL in a browser shows `404 Not Found`. That's expected; `/healthz` returns `ok` when the service is up. + +## Get running + +The [Quickstart](docs/0_QUICKSTART.md) has every step in detail, including how to add your own repositories. You need a Render workspace with Sandboxes access in the Oregon region, a [Render API key](docs/0_QUICKSTART.md#set-secrets), and a model API key (Anthropic by default). The snapshot cron runs on a paid Standard instance. + +1. Fork this repository. Edit `config.ts` to list your repositories and `prompt.md` to describe your product. Commit and push. For private GitHub repositories, [create a read-only token](https://github.com/settings/personal-access-tokens/new?name=Codebase+Researcher&contents=read). +2. In the Render Dashboard, create a Blueprint from your fork and paste the secrets it asks for. +3. Trigger the `codebase-researcher-snapshots` cron job once to build the first snapshot (about 5 minutes). +4. Copy `API_TOKEN` from the **Environment** page of the `codebase-researcher-api` service, then ask a question from your checkout: + + ```bash + pnpm install + API_URL=https://.onrender.com API_TOKEN=... \ + pnpm ask "How does the CLI pick a workspace in CI?" + ``` + + An investigation usually takes one to two minutes. [Ask your first question](docs/0_QUICKSTART.md#ask-your-first-question) shows the same request with curl. + +To try it on your machine first, without a Sandbox or a model key, clone a couple of repositories into `.scratch/repos/` and run the workflow and the API locally with the runner's `--faux` mode, which follows a fixed script instead of calling a model. [Run it locally](docs/0_QUICKSTART.md#run-it-locally) has the commands. + +## What you get + +Every answer has the same structure: a short answer, a confidence level, what was observed in code, what was inferred, what was not established, file-and-line evidence, a suggested owner, and an escalation-ready question for engineering. When the question is ambiguous, the agent asks one clarifying question instead, and you answer it as a follow-up. + +### Example answer + +A completed poll returns the answer under `reply.result`: ```json { @@ -47,7 +74,7 @@ An answer about code is only useful if you can check it. A model can sound confi A reader can follow each link to the exact lines and decide for themselves. -### What it demonstrates on Render +### Built on Render - **Sandboxes:** one fresh, isolated Sandbox per question, restored from a snapshot that a privileged builder Sandbox prepared once. The investigation Sandbox has no git credentials and, by default, can reach only the model provider's API. - **Workflows:** each question is a durable workflow run with subtasks, per-attempt timeouts, and retries for infrastructure failures. The run is the record of the question and its answer, which is also how a follow-up recovers the conversation. @@ -76,28 +103,6 @@ A reader can follow each link to the exact lines and decide for themselves. The architecture guide's [trust boundaries](docs/2_ARCHITECTURE.md#trust-boundaries) section explains these properties and the residual risks. Runtime hardening changes reach investigation Sandboxes only after a successful snapshot refresh; deploying the services alone does not update existing snapshots. -## Get running - -You need a Render workspace with Sandboxes access in the Oregon region, a -Render API key, and a model API key (Anthropic by default). The snapshot cron -runs on a paid Standard instance; see the -[prerequisites](docs/0_QUICKSTART.md#prerequisites) before deploying. - -1. Fork this repository. Edit `config.ts` to list your repositories and `prompt.md` to describe your product. Commit and push. -2. In the Render Dashboard, create a Blueprint from your fork and paste the secrets it asks for. -3. Trigger the `codebase-researcher-snapshots` cron job once to build the first snapshot (about 5 minutes). -4. Ask a question from your checkout, using the API service's URL and its generated `API_TOKEN`: - - ```bash - pnpm install - API_URL=https://.onrender.com API_TOKEN=... \ - pnpm ask "How does the CLI pick a workspace in CI?" - ``` - - An investigation usually takes one to two minutes. [Ask your first question](docs/0_QUICKSTART.md#ask-your-first-question) shows the same request with curl. - -To try it on your machine first, without a Sandbox or a model key, clone a couple of repositories into `.scratch/repos/` and run the workflow and the API locally with the runner's `--faux` mode, which follows a fixed script instead of calling a model. [Run it locally](docs/0_QUICKSTART.md#run-it-locally) has the commands. - ## Learn more - [docs/0_QUICKSTART.md](docs/0_QUICKSTART.md): deploy, ask a question, follow up, develop locally, and operate the service. diff --git a/docs/0_QUICKSTART.md b/docs/0_QUICKSTART.md index fdb5317..ef6d83d 100644 --- a/docs/0_QUICKSTART.md +++ b/docs/0_QUICKSTART.md @@ -13,7 +13,9 @@ continue to [2_ARCHITECTURE.md](2_ARCHITECTURE.md). `render.yaml`. The API runs on the free instance type. - A dedicated, spend-capped **model API key**. Anthropic is the default; see [Choose a model](1_EXTENSION.md#choose-a-model). -- For private repositories, a read-only token for their Git host. +- For private repositories, a read-only token for their Git host. For GitHub, + [create a fine-grained token](https://github.com/settings/personal-access-tokens/new?name=Codebase+Researcher&contents=read) + with read access to the repositories' contents. - Node (the version in `.node-version`) and pnpm 11 on your machine, to run `pnpm ask`. @@ -41,7 +43,8 @@ and [prompt](1_EXTENSION.md#customize-the-agent). Create a Blueprint from the fork in the Render Dashboard. [`render.yaml`](../render.yaml) creates one project with three services: -- `codebase-researcher-api`: the HTTP API. +- `codebase-researcher-api`: the HTTP API. It has no UI: its only routes are + `/healthz` and `/v1/*`, so opening its URL in a browser shows `404 Not Found`. - `codebase-researcher-workflow`: the durable question workflow. - `codebase-researcher-snapshots`: the hourly snapshot refresh cron. @@ -60,6 +63,15 @@ blank `sync: false` value counts as unset. | Snapshot cron | `RENDER_WORKSPACE_ID` | The key can access multiple workspaces | | Snapshot cron | `GITHUB_TOKEN` | Reading private GitHub repositories | +If you use [scoped API keys](https://render.com/docs/scoped-api-keys), each +service's `RENDER_API_KEY` needs only these permissions in the workspace: + +| Service | Permissions | +|---|---| +| API | **Workflows:** Read. **Run workflow tasks:** Run. | +| Workflow | **Sandbox lifecycle and snapshots:** Read & write. **Sandbox execution and file transfer:** Execute. | +| Snapshot cron | **Sandbox lifecycle and snapshots:** Read & write. **Sandbox execution and file transfer:** Execute. | + The Blueprint generates separate `API_TOKEN` and `QUESTION_ID_SECRET` values on the API service and wires `WORKFLOW_SLUG` from the workflow service into the API. Share only `API_TOKEN` with callers. Keep `QUESTION_ID_SECRET` stable @@ -80,11 +92,14 @@ repository snapshot is available" reply. ## Ask your first question -Copy `API_TOKEN` from the API service and use its public URL: +In the Render Dashboard, open the `codebase-researcher-api` service, go to its +**Environment** page, and reveal and copy `API_TOKEN`. The service's public URL +is at the top of the page. Confirm the API is up; it returns `ok`: ```bash export API_TOKEN=... export API_URL=https://codebase-researcher-api.onrender.com +curl "$API_URL/healthz" pnpm install pnpm ask "How does the CLI pick a workspace in CI?" @@ -144,6 +159,37 @@ Over HTTP, send `"follow_up_to": ""` with the new `question`. The API reads the earlier run's conversation and reply and starts a new investigation with the whole exchange, so the agent sees what it already asked or answered. +## Add a repository + +To add a repository to a running deployment and confirm the agent can read it: + +1. Add the repository to `repos` in [`config.ts`](../config.ts). See + [Configure the repository corpus](1_EXTENSION.md#configure-the-repository-corpus) + for branches and non-GitHub hosts. +2. For a private repository, set a read-only token on the + `codebase-researcher-snapshots` cron, such as `GITHUB_TOKEN`. Only the cron + clones, so only it needs the token. +3. Commit and push. +4. Wait for the cron to deploy that commit. A `config.ts` change only takes + effect once the cron's build of that commit is live: open the cron's + **Deploys** page and check that a deploy of your commit finished. A run + before that rebuilds the old list of repositories. +5. On the cron's **Runs** page, click **Trigger Run**, or wait for the hourly + schedule. +6. In the cron's logs, find the `snapshot_refresh` line. Check that `ok` is + `true`, that `commit` is your commit, and that `repositories` includes the + new one. +7. Ask a question about the new repository with `pnpm ask` and check that the + evidence links point to it. + +If one repository can't be cloned, the whole build fails and the previous +snapshot stays live. The `error` field on `snapshot_refresh` names the +repository. Common causes are a missing or under-scoped token, a misspelled +repository name, or a corpus larger than the Sandbox's 10 GB disk. + +If no deploy of your commit appears on the cron's **Deploys** page, click +**Manual Deploy > Deploy latest commit**. + ## Run it locally Local development uses the real workflow, runner, tools, and citation checks, @@ -225,7 +271,7 @@ All services emit one JSON object per line. The most useful events are: | `investigation_failed` | Workflow | The agent ran but did not submit an accepted result | | `workflow_misconfigured` | Workflow | Model or Sandbox configuration is invalid | | `deliver_failed` | Workflow | Callback retries were exhausted | -| `snapshot_refresh` | Cron | Snapshot build result, repository count, and duration | +| `snapshot_refresh` | Cron | Snapshot build result, build commit, repositories, and duration | Every reply and workflow-level failure response includes a trace ID. Use it to find the task run in the Dashboard. Alert on consecutive `snapshot_refresh` @@ -250,12 +296,14 @@ events with `"ok": false`; a failed refresh leaves the previous snapshot availab the wrong workspace ID, or a failed `verify.sh` check. - **Investigation service is misconfigured:** inspect `workflow_misconfigured`. Verify `MODEL_PROVIDER`, `MODEL`, - `RENDER_API_KEY`, and `RENDER_WORKSPACE_ID`. + `RENDER_API_KEY`, and `RENDER_WORKSPACE_ID`. If Render rejected the egress + allow-list, check `SANDBOX_ALLOWED_DOMAINS`, or set + `SANDBOX_NETWORK_POLICY=allow-all` if your workspace doesn't support + allow-lists. - **Agent stopped without submitting:** inspect `investigation_failed`. Authentication failures usually mean the model key is missing or invalid. - **Infrastructure retries exhausted:** inspect failed `investigate` task - runs. Restore timeouts, Sandbox verification, or a rejected network policy - are common causes. + runs. Restore timeouts and Sandbox verification are common causes. - **Callback never arrives:** inspect `deliver_failed` and the `deliver` subtask's attempts. The receiver must return 2xx within 10 seconds. - **First request is slow:** the free API instance spun down after 15 idle diff --git a/docs/1_EXTENSION.md b/docs/1_EXTENSION.md index e659609..1c6c4ba 100644 --- a/docs/1_EXTENSION.md +++ b/docs/1_EXTENSION.md @@ -40,7 +40,9 @@ values for local development and tests. Repository, runner, prompt, and `sandbox/` changes become available only after the snapshot cron builds a new snapshot. All checkouts and runtime -artifacts must fit within the Sandbox's 10 GB. +artifacts must fit within the Sandbox's 10 GB. To add a repository to a +running deployment and confirm it's searchable, follow +[Add a repository](0_QUICKSTART.md#add-a-repository). ### Private repositories diff --git a/docs/2_ARCHITECTURE.md b/docs/2_ARCHITECTURE.md index a369380..865c967 100644 --- a/docs/2_ARCHITECTURE.md +++ b/docs/2_ARCHITECTURE.md @@ -11,7 +11,8 @@ The Blueprint in [`render.yaml`](../render.yaml) creates three services: - **API** (`codebase-researcher-api`) accepts HTTP questions, rebuilds the conversation for follow-ups, starts durable workflow runs, and serves their - results. It holds no state of its own. + results. It holds no state of its own and has no UI: its only routes are + `/healthz` and `/v1/*`, so `/` returns 404. - **Workflow** (`codebase-researcher-workflow`) orchestrates each question, restores a fresh Sandbox, runs the agent inside it, validates the result, and optionally delivers a callback. diff --git a/render.yaml b/render.yaml index a6aece0..a85d1bb 100644 --- a/render.yaml +++ b/render.yaml @@ -9,6 +9,7 @@ projects: environments: - name: production services: + # HTTP API with no UI: `/` returns 404. Check `/healthz` (returns ok) or ask with `pnpm ask`. - type: web name: codebase-researcher-api runtime: node diff --git a/src/cron.ts b/src/cron.ts index 6c38f8a..7802c82 100644 --- a/src/cron.ts +++ b/src/cron.ts @@ -11,28 +11,32 @@ const KEEP_SNAPSHOTS = 2; export type RefreshDeps = { snapshotName: string; - /** Builds the snapshot and prunes older ones. */ - rebuild: () => Promise<{ id: string; repos: number }>; + /** Builds the snapshot and prunes older ones; returns the repositories it cloned. */ + rebuild: () => Promise<{ id: string; repos: string[] }>; + /** The commit this cron build runs. A config.ts change only reaches snapshots once its build is live. */ + commit?: string; log: Log; }; -type RefreshBase = { event: "snapshot_refresh"; snapshot_name: string; duration_ms: number }; +type RefreshBase = { event: "snapshot_refresh"; snapshot_name: string; commit: string | null; duration_ms: number }; type RefreshOutcome = - | { ok: true; snapshot_id: string; repos: number } + | { ok: true; snapshot_id: string; repos: number; repositories: string[] } | { ok: false; snapshot_id: null; repos: 0; error: string }; export type RefreshResult = RefreshBase & RefreshOutcome; /** Never throws: logs one `snapshot_refresh` line either way. */ -export async function refreshSnapshot({ snapshotName, rebuild, log }: RefreshDeps): Promise { +export async function refreshSnapshot({ snapshotName, rebuild, commit, log }: RefreshDeps): Promise { const start = Date.now(); let outcome: RefreshOutcome; try { const built = await rebuild(); - outcome = { ok: true, snapshot_id: built.id, repos: built.repos }; + outcome = { ok: true, snapshot_id: built.id, repos: built.repos.length, repositories: built.repos }; } catch (err) { outcome = { ok: false, snapshot_id: null, repos: 0, error: errorMessage(err) }; } - const result: RefreshResult = { event: "snapshot_refresh", snapshot_name: snapshotName, duration_ms: Date.now() - start, ...outcome }; + const result: RefreshResult = { + event: "snapshot_refresh", snapshot_name: snapshotName, commit: commit ?? null, duration_ms: Date.now() - start, ...outcome, + }; if (result.ok) log.info("snapshot refreshed", result); else log.error("snapshot refresh failed", result); return result; @@ -43,6 +47,7 @@ if (process.argv[1] && import.meta.url === pathToFileURL(realpathSync(process.ar const log = createLog(); const result = await refreshSnapshot({ snapshotName, + commit: process.env.RENDER_GIT_COMMIT, log, rebuild: async () => rebuildSnapshot(SandboxClient.fromConfig(await loadRenderConfig(), log), { diff --git a/src/sandbox/index.ts b/src/sandbox/index.ts index e4f7b7d..ac96830 100644 --- a/src/sandbox/index.ts +++ b/src/sandbox/index.ts @@ -5,7 +5,7 @@ import { SandboxClient } from "./render/client.ts"; import { loadRenderConfig } from "./render/config.ts"; import { openRenderSandbox } from "./render/sandbox.ts"; -export { SnapshotUnavailableError, type OpenSandbox, type RunInvestigationResult, type Sandbox } from "./interface.ts"; +export { SandboxMisconfiguredError, SnapshotUnavailableError, type OpenSandbox, type RunInvestigationResult, type Sandbox } from "./interface.ts"; /** * Loads sandbox config and returns a way to open investigation sandboxes. Selects by `SANDBOX_PROVIDER` diff --git a/src/sandbox/interface.ts b/src/sandbox/interface.ts index c394535..35f648f 100644 --- a/src/sandbox/interface.ts +++ b/src/sandbox/interface.ts @@ -70,6 +70,11 @@ export class SnapshotUnavailableError extends Error { } } +/** Thrown when the provider rejects the Sandbox settings themselves, such as the egress policy. Retrying won't help. */ +export class SandboxMisconfiguredError extends Error { + override name = "SandboxMisconfiguredError"; +} + const STDERR_TAIL_CHARS = 4096; export const appendStderrTail = (tail: string, chunk: string): string => (tail + chunk).slice(-STDERR_TAIL_CHARS); diff --git a/src/sandbox/render/sandbox.ts b/src/sandbox/render/sandbox.ts index d82b8ca..e893238 100644 --- a/src/sandbox/render/sandbox.ts +++ b/src/sandbox/render/sandbox.ts @@ -5,6 +5,7 @@ import { z } from "zod"; import { RepoManifest } from "../../manifest.ts"; import { errorMessage } from "../../util.ts"; import { + SandboxMisconfiguredError, SnapshotUnavailableError, appendStderrTail, parseRunnerEvents, @@ -110,7 +111,7 @@ export async function openRenderSandbox(client: SandboxClient, snapshotName: str } if (signal.aborted) throw timedOut(err); if (policy.type === "allow-list" && err instanceof ClientError && [400, 422].includes(err.statusCode)) { - throw new Error( + throw new SandboxMisconfiguredError( `Render rejected HTTPS egress allow-list [${policyRules(policy).join(", ")}]: ${err.message}`, { cause: err }, ); diff --git a/src/sandbox/render/snapshot.ts b/src/sandbox/render/snapshot.ts index 53301ce..567d87a 100644 --- a/src/sandbox/render/snapshot.ts +++ b/src/sandbox/render/snapshot.ts @@ -19,11 +19,11 @@ export type BuiltSnapshot = { id: string; sandboxGroupId: string }; export async function rebuildSnapshot( client: SandboxClient, opts: { profile: RepoProfile; snapshotName: string; keep: number }, -): Promise<{ id: string; repos: number }> { +): Promise<{ id: string; repos: string[] }> { const heads = await resolveHeads(opts.profile, client.cfg.hostCredentials); const built = await buildSnapshot(client, { profile: opts.profile, heads, snapshotName: opts.snapshotName }); await pruneSnapshots(client, built, opts.snapshotName, opts.keep); - return { id: built.id, repos: heads.length }; + return { id: built.id, repos: heads.map((h) => h.repository) }; } /** Resolves the current default-branch (or configured branch) commit of each repo with `git ls-remote`. */ diff --git a/src/tasks.ts b/src/tasks.ts index d02c212..c02664d 100644 --- a/src/tasks.ts +++ b/src/tasks.ts @@ -10,7 +10,7 @@ import { } from "./investigation.ts"; import type { RepoManifest } from "./manifest.ts"; import { RUNNER_EXIT, RunnerInput, type RunnerEvent, type Usage } from "./runner/protocol.ts"; -import { SnapshotUnavailableError, type OpenSandbox, type RunInvestigationResult, type Sandbox } from "./sandbox/index.ts"; +import { SandboxMisconfiguredError, SnapshotUnavailableError, type OpenSandbox, type RunInvestigationResult, type Sandbox } from "./sandbox/index.ts"; import { getBuiltinModel } from "@earendil-works/pi-ai/providers/all"; import { signCallback } from "./callback.ts"; import type { Log } from "./log.ts"; @@ -294,6 +294,10 @@ export async function investigateTask( sandbox = await openSandbox(config.snapshotName, { timeoutMs: config.sandboxProvisionTimeoutMs }); } catch (err) { if (err instanceof SnapshotUnavailableError) return { kind: "failed", code: err.notReady ? "snapshot_not_ready" : "no_snapshot" }; + if (err instanceof SandboxMisconfiguredError) { + log.warn("sandbox settings rejected", { event: "workflow_misconfigured", err }); + return { kind: "failed", code: "misconfigured" }; + } throw err; } log.info("sandbox ready", { sandbox_id: sandbox.id, ...sandbox.info }); diff --git a/test/cron.test.ts b/test/cron.test.ts index 1fbd6cb..e3d405b 100644 --- a/test/cron.test.ts +++ b/test/cron.test.ts @@ -12,9 +12,10 @@ function capture() { describe("refreshSnapshot", () => { it("logs one info snapshot_refresh line with the built snapshot", async () => { const { lines, log } = capture(); - const result = await refreshSnapshot({ snapshotName: SNAPSHOT_NAME, rebuild: async () => ({ id: "snp-1", repos: 3 }), log }); + const repos = ["render-oss/cli", "render-oss/sdk", "render-oss/skills"]; + const result = await refreshSnapshot({ snapshotName: SNAPSHOT_NAME, rebuild: async () => ({ id: "snp-1", repos }), commit: "abc1234", log }); - expect(result).toMatchObject({ ok: true, snapshot_id: "snp-1", repos: 3 }); + expect(result).toMatchObject({ ok: true, snapshot_id: "snp-1", repos: 3, repositories: repos, commit: "abc1234" }); expect(lines).toEqual([ expect.objectContaining({ level: "info", @@ -22,7 +23,9 @@ describe("refreshSnapshot", () => { ok: true, snapshot_id: "snp-1", snapshot_name: SNAPSHOT_NAME, + commit: "abc1234", repos: 3, + repositories: repos, duration_ms: expect.any(Number), }), ]); @@ -43,6 +46,7 @@ describe("refreshSnapshot", () => { ok: false, snapshot_id: null, snapshot_name: SNAPSHOT_NAME, + commit: null, repos: 0, error: "clone failed", duration_ms: expect.any(Number), diff --git a/test/sandbox-render.test.ts b/test/sandbox-render.test.ts index 350b6c8..a12889d 100644 --- a/test/sandbox-render.test.ts +++ b/test/sandbox-render.test.ts @@ -4,7 +4,7 @@ import { AbortError, RenderError, SandboxSnapshotNotFoundError, SandboxSnapshotN import type { SandboxExecEvent } from "@renderinc/sdk/experimental"; import { beforeEach, describe, expect, it } from "vitest"; import { configureSandboxes } from "../src/sandbox/index.ts"; -import { SnapshotUnavailableError } from "../src/sandbox/interface.ts"; +import { SandboxMisconfiguredError, SnapshotUnavailableError } from "../src/sandbox/interface.ts"; import { ClientError } from "@renderinc/sdk"; import { SandboxClient, type SandboxApi } from "../src/sandbox/render/client.ts"; import { loadRenderConfig, type NetworkPolicy, type RenderConfig } from "../src/sandbox/render/config.ts"; @@ -382,6 +382,11 @@ describe("openRenderSandbox", () => { expect(err).toMatchObject({ notReady }); }); + it("throws SandboxMisconfiguredError when Render rejects the allow-list, so it isn't retried", async () => { + fake.createError = () => new ClientError("invalid networkPolicy", 422); + await expect(openRenderSandbox(client(), NAME, OPEN)).rejects.toBeInstanceOf(SandboxMisconfiguredError); + }); + it("refuses and terminates a sandbox where verify.sh couldn't confirm the runner is non-dumpable", async () => { fake.exec_ = dumpableExec; await expect(openRenderSandbox(client(), NAME, OPEN)).rejects.toThrow("non-dumpable"); diff --git a/test/tasks.test.ts b/test/tasks.test.ts index c15e84f..606e05b 100644 --- a/test/tasks.test.ts +++ b/test/tasks.test.ts @@ -5,7 +5,7 @@ import type { Evidence, InvestigationResult } from "../src/investigation.ts"; import { createLog } from "../src/log.ts"; import type { AnswerQuestionInput } from "../src/question.ts"; import type { RunnerEvent } from "../src/runner/protocol.ts"; -import type { OpenSandbox } from "../src/sandbox/interface.ts"; +import { SandboxMisconfiguredError, type OpenSandbox } from "../src/sandbox/interface.ts"; import { answerQuestion, deliverCallback, investigateTask, type DeliverInput, type InvestigateInput } from "../src/tasks.ts"; import { DELIVER_MAX_RETRIES, @@ -147,6 +147,15 @@ describe("answerQuestion", () => { }, reason: "misconfigured", }, + { + name: "Render rejects the Sandbox egress policy", + arrange: (h) => { + h.configureSandboxes = async () => async () => { + throw new SandboxMisconfiguredError("Render rejected HTTPS egress allow-list"); + }; + }, + reason: "misconfigured", + }, ])("replies with a user-safe failure when $name", async ({ env, arrange, reason }) => { const h = harness(env); arrange(h);