Docs for first deploy and adding repositories; fail fast on rejected egress

README.md

@@ -1,12 +1,39 @@
 # codebase-researcher
 
-A template for getting evidence-backed answers to questions about how your code behaves. Post a question to an HTTP API, or run `pnpm ask "..."`. An AI coding agent reads your repositories in a fresh, locked-down Render Sandbox and answers with file-and-line citations that are checked against the exact commit it read.
+Ask how your code behaves and get an answer with file-and-line citations, checked against the exact commit an AI agent read in a locked-down Render Sandbox.
 
 [![Deploy to Render](https://render.com/images/deploy-to-render-button.svg)](https://render.com/deploy?repo=https://github.com/render-examples/codebase-researcher)
 
 The button deploys this repository as is, investigating its example corpus: three public repositories (the Render CLI, SDK, and agent skills). To point it at your own code, fork first and deploy your fork.
 
-Every answer has the same structure: a short answer, a confidence level, what was observed in code, what was inferred, what was not established, file-and-line evidence, a suggested owner, and an escalation-ready question for engineering. When the question is ambiguous, the agent asks one clarifying question instead, and you answer it as a follow-up. A completed poll returns the answer under `reply.result`:
+The researcher is an HTTP API with no UI. You ask questions with `pnpm ask` or `curl`, so opening the service URL in a browser shows `404 Not Found`. That's expected; `/healthz` returns `ok` when the service is up.
+
+## Get running
+
+The [Quickstart](docs/0_QUICKSTART.md) has every step in detail, including how to add your own repositories. You need a Render workspace with Sandboxes access in the Oregon region, a [Render API key](docs/0_QUICKSTART.md#set-secrets), and a model API key (Anthropic by default). The snapshot cron runs on a paid Standard instance.
+
+1. Fork this repository. Edit `config.ts` to list your repositories and `prompt.md` to describe your product. Commit and push. For private GitHub repositories, [create a read-only token](https://github.com/settings/personal-access-tokens/new?name=Codebase+Researcher&contents=read).
+2. In the Render Dashboard, create a Blueprint from your fork and paste the secrets it asks for.
+3. Trigger the `codebase-researcher-snapshots` cron job once to build the first snapshot (about 5 minutes).
+4. Copy `API_TOKEN` from the **Environment** page of the `codebase-researcher-api` service, then ask a question from your checkout:
+
+   ```bash
+   pnpm install
+   API_URL=https://<your-api-service>.onrender.com API_TOKEN=... \
+     pnpm ask "How does the CLI pick a workspace in CI?"
+   ```
+
+   An investigation usually takes one to two minutes. [Ask your first question](docs/0_QUICKSTART.md#ask-your-first-question) shows the same request with curl.
+
+To try it on your machine first, without a Sandbox or a model key, clone a couple of repositories into `.scratch/repos/` and run the workflow and the API locally with the runner's `--faux` mode, which follows a fixed script instead of calling a model. [Run it locally](docs/0_QUICKSTART.md#run-it-locally) has the commands.
+
+## What you get
+
+Every answer has the same structure: a short answer, a confidence level, what was observed in code, what was inferred, what was not established, file-and-line evidence, a suggested owner, and an escalation-ready question for engineering. When the question is ambiguous, the agent asks one clarifying question instead, and you answer it as a follow-up.
+
+### Example answer
+
+A completed poll returns the answer under `reply.result`:
 
 ```json
 {
@@ -47,7 +74,7 @@ An answer about code is only useful if you can check it. A model can sound confi
 
 A reader can follow each link to the exact lines and decide for themselves.
 
-### What it demonstrates on Render
+### Built on Render
 
 - **Sandboxes:** one fresh, isolated Sandbox per question, restored from a snapshot that a privileged builder Sandbox prepared once. The investigation Sandbox has no git credentials and, by default, can reach only the model provider's API.
 - **Workflows:** each question is a durable workflow run with subtasks, per-attempt timeouts, and retries for infrastructure failures. The run is the record of the question and its answer, which is also how a follow-up recovers the conversation.
@@ -76,28 +103,6 @@ A reader can follow each link to the exact lines and decide for themselves.
 The architecture guide's [trust boundaries](docs/2_ARCHITECTURE.md#trust-boundaries)
 section explains these properties and the residual risks. Runtime hardening changes reach investigation Sandboxes only after a successful snapshot refresh; deploying the services alone does not update existing snapshots.
 
-## Get running
-
-You need a Render workspace with Sandboxes access in the Oregon region, a
-Render API key, and a model API key (Anthropic by default). The snapshot cron
-runs on a paid Standard instance; see the
-[prerequisites](docs/0_QUICKSTART.md#prerequisites) before deploying.
-
-1. Fork this repository. Edit `config.ts` to list your repositories and `prompt.md` to describe your product. Commit and push.
-2. In the Render Dashboard, create a Blueprint from your fork and paste the secrets it asks for.
-3. Trigger the `codebase-researcher-snapshots` cron job once to build the first snapshot (about 5 minutes).
-4. Ask a question from your checkout, using the API service's URL and its generated `API_TOKEN`:
-
-   ```bash
-   pnpm install
-   API_URL=https://<your-api-service>.onrender.com API_TOKEN=... \
-     pnpm ask "How does the CLI pick a workspace in CI?"
-   ```
-
-   An investigation usually takes one to two minutes. [Ask your first question](docs/0_QUICKSTART.md#ask-your-first-question) shows the same request with curl.
-
-To try it on your machine first, without a Sandbox or a model key, clone a couple of repositories into `.scratch/repos/` and run the workflow and the API locally with the runner's `--faux` mode, which follows a fixed script instead of calling a model. [Run it locally](docs/0_QUICKSTART.md#run-it-locally) has the commands.
-
 ## Learn more
 
 - [docs/0_QUICKSTART.md](docs/0_QUICKSTART.md): deploy, ask a question, follow up, develop locally, and operate the service.

docs/0_QUICKSTART.md

@@ -13,7 +13,9 @@ continue to [2_ARCHITECTURE.md](2_ARCHITECTURE.md).
   `render.yaml`. The API runs on the free instance type.
 - A dedicated, spend-capped **model API key**. Anthropic is the default; see
   [Choose a model](1_EXTENSION.md#choose-a-model).
-- For private repositories, a read-only token for their Git host.
+- For private repositories, a read-only token for their Git host. For GitHub,
+  [create a fine-grained token](https://github.com/settings/personal-access-tokens/new?name=Codebase+Researcher&contents=read)
+  with read access to the repositories' contents.
 - Node (the version in `.node-version`) and pnpm 11 on your machine, to run
   `pnpm ask`.
 
@@ -41,7 +43,8 @@ and [prompt](1_EXTENSION.md#customize-the-agent).
 Create a Blueprint from the fork in the Render Dashboard. [`render.yaml`](../render.yaml)
 creates one project with three services:
 
-- `codebase-researcher-api`: the HTTP API.
+- `codebase-researcher-api`: the HTTP API. It has no UI: its only routes are
+  `/healthz` and `/v1/*`, so opening its URL in a browser shows `404 Not Found`.
 - `codebase-researcher-workflow`: the durable question workflow.
 - `codebase-researcher-snapshots`: the hourly snapshot refresh cron.
 
@@ -60,6 +63,15 @@ blank `sync: false` value counts as unset.
 | Snapshot cron | `RENDER_WORKSPACE_ID` | The key can access multiple workspaces |
 | Snapshot cron | `GITHUB_TOKEN` | Reading private GitHub repositories |
 
+If you use [scoped API keys](https://render.com/docs/scoped-api-keys), each
+service's `RENDER_API_KEY` needs only these permissions in the workspace:
+
+| Service | Permissions |
+|---|---|
+| API | **Workflows:** Read. **Run workflow tasks:** Run. |
+| Workflow | **Sandbox lifecycle and snapshots:** Read & write. **Sandbox execution and file transfer:** Execute. |
+| Snapshot cron | **Sandbox lifecycle and snapshots:** Read & write. **Sandbox execution and file transfer:** Execute. |
+
 The Blueprint generates separate `API_TOKEN` and `QUESTION_ID_SECRET` values
 on the API service and wires `WORKFLOW_SLUG` from the workflow service into
 the API. Share only `API_TOKEN` with callers. Keep `QUESTION_ID_SECRET` stable
@@ -80,11 +92,14 @@ repository snapshot is available" reply.
 
 ## Ask your first question
 
-Copy `API_TOKEN` from the API service and use its public URL:
+In the Render Dashboard, open the `codebase-researcher-api` service, go to its
+**Environment** page, and reveal and copy `API_TOKEN`. The service's public URL
+is at the top of the page. Confirm the API is up; it returns `ok`:
 
 ```bash
 export API_TOKEN=...
 export API_URL=https://codebase-researcher-api.onrender.com
+curl "$API_URL/healthz"
 
 pnpm install
 pnpm ask "How does the CLI pick a workspace in CI?"
@@ -144,6 +159,37 @@ Over HTTP, send `"follow_up_to": "<id>"` with the new `question`. The API
 reads the earlier run's conversation and reply and starts a new investigation
 with the whole exchange, so the agent sees what it already asked or answered.
 
+## Add a repository
+
+To add a repository to a running deployment and confirm the agent can read it:
+
+1. Add the repository to `repos` in [`config.ts`](../config.ts). See
+   [Configure the repository corpus](1_EXTENSION.md#configure-the-repository-corpus)
+   for branches and non-GitHub hosts.
+2. For a private repository, set a read-only token on the
+   `codebase-researcher-snapshots` cron, such as `GITHUB_TOKEN`. Only the cron
+   clones, so only it needs the token.
+3. Commit and push.
+4. Wait for the cron to deploy that commit. A `config.ts` change only takes
+   effect once the cron's build of that commit is live: open the cron's
+   **Deploys** page and check that a deploy of your commit finished. A run
+   before that rebuilds the old list of repositories.
+5. On the cron's **Runs** page, click **Trigger Run**, or wait for the hourly
+   schedule.
+6. In the cron's logs, find the `snapshot_refresh` line. Check that `ok` is
+   `true`, that `commit` is your commit, and that `repositories` includes the
+   new one.
+7. Ask a question about the new repository with `pnpm ask` and check that the
+   evidence links point to it.
+
+If one repository can't be cloned, the whole build fails and the previous
+snapshot stays live. The `error` field on `snapshot_refresh` names the
+repository. Common causes are a missing or under-scoped token, a misspelled
+repository name, or a corpus larger than the Sandbox's 10 GB disk.
+
+If no deploy of your commit appears on the cron's **Deploys** page, click
+**Manual Deploy > Deploy latest commit**.
+
 ## Run it locally
 
 Local development uses the real workflow, runner, tools, and citation checks,
@@ -225,7 +271,7 @@ All services emit one JSON object per line. The most useful events are:
 | `investigation_failed` | Workflow | The agent ran but did not submit an accepted result |
 | `workflow_misconfigured` | Workflow | Model or Sandbox configuration is invalid |
 | `deliver_failed` | Workflow | Callback retries were exhausted |
-| `snapshot_refresh` | Cron | Snapshot build result, repository count, and duration |
+| `snapshot_refresh` | Cron | Snapshot build result, build commit, repositories, and duration |
 
 Every reply and workflow-level failure response includes a trace ID. Use it
 to find the task run in the Dashboard. Alert on consecutive `snapshot_refresh`
@@ -250,12 +296,14 @@ events with `"ok": false`; a failed refresh leaves the previous snapshot availab
   the wrong workspace ID, or a failed `verify.sh` check.
 - **Investigation service is misconfigured:** inspect
   `workflow_misconfigured`. Verify `MODEL_PROVIDER`, `MODEL`,
-  `RENDER_API_KEY`, and `RENDER_WORKSPACE_ID`.
+  `RENDER_API_KEY`, and `RENDER_WORKSPACE_ID`. If Render rejected the egress
+  allow-list, check `SANDBOX_ALLOWED_DOMAINS`, or set
+  `SANDBOX_NETWORK_POLICY=allow-all` if your workspace doesn't support
+  allow-lists.
 - **Agent stopped without submitting:** inspect `investigation_failed`.
   Authentication failures usually mean the model key is missing or invalid.
 - **Infrastructure retries exhausted:** inspect failed `investigate` task
-  runs. Restore timeouts, Sandbox verification, or a rejected network policy
-  are common causes.
+  runs. Restore timeouts and Sandbox verification are common causes.
 - **Callback never arrives:** inspect `deliver_failed` and the `deliver`
   subtask's attempts. The receiver must return 2xx within 10 seconds.
 - **First request is slow:** the free API instance spun down after 15 idle

docs/1_EXTENSION.md

@@ -40,7 +40,9 @@ values for local development and tests.
 
 Repository, runner, prompt, and `sandbox/` changes become available
 only after the snapshot cron builds a new snapshot. All checkouts and runtime
-artifacts must fit within the Sandbox's 10 GB.
+artifacts must fit within the Sandbox's 10 GB. To add a repository to a
+running deployment and confirm it's searchable, follow
+[Add a repository](0_QUICKSTART.md#add-a-repository).
 
 ### Private repositories
 

docs/2_ARCHITECTURE.md

@@ -11,7 +11,8 @@ The Blueprint in [`render.yaml`](../render.yaml) creates three services:
 
 - **API** (`codebase-researcher-api`) accepts HTTP questions, rebuilds the
   conversation for follow-ups, starts durable workflow runs, and serves their
-  results. It holds no state of its own.
+  results. It holds no state of its own and has no UI: its only routes are
+  `/healthz` and `/v1/*`, so `/` returns 404.
 - **Workflow** (`codebase-researcher-workflow`) orchestrates each question,
   restores a fresh Sandbox, runs the agent inside it, validates the result,
   and optionally delivers a callback.

render.yaml

@@ -9,6 +9,7 @@ projects:
     environments:
       - name: production
         services:
+          # HTTP API with no UI: `/` returns 404. Check `/healthz` (returns ok) or ask with `pnpm ask`.
           - type: web
             name: codebase-researcher-api
             runtime: node

src/cron.ts

@@ -11,28 +11,32 @@ const KEEP_SNAPSHOTS = 2;
 
 export type RefreshDeps = {
   snapshotName: string;
-  /** Builds the snapshot and prunes older ones. */
-  rebuild: () => Promise<{ id: string; repos: number }>;
+  /** Builds the snapshot and prunes older ones; returns the repositories it cloned. */
+  rebuild: () => Promise<{ id: string; repos: string[] }>;
+  /** The commit this cron build runs. A config.ts change only reaches snapshots once its build is live. */
+  commit?: string;
   log: Log;
 };
 
-type RefreshBase = { event: "snapshot_refresh"; snapshot_name: string; duration_ms: number };
+type RefreshBase = { event: "snapshot_refresh"; snapshot_name: string; commit: string | null; duration_ms: number };
 type RefreshOutcome =
-  | { ok: true; snapshot_id: string; repos: number }
+  | { ok: true; snapshot_id: string; repos: number; repositories: string[] }
   | { ok: false; snapshot_id: null; repos: 0; error: string };
 export type RefreshResult = RefreshBase & RefreshOutcome;
 
 /** Never throws: logs one `snapshot_refresh` line either way. */
-export async function refreshSnapshot({ snapshotName, rebuild, log }: RefreshDeps): Promise<RefreshResult> {
+export async function refreshSnapshot({ snapshotName, rebuild, commit, log }: RefreshDeps): Promise<RefreshResult> {
   const start = Date.now();
   let outcome: RefreshOutcome;
   try {
     const built = await rebuild();
-    outcome = { ok: true, snapshot_id: built.id, repos: built.repos };
+    outcome = { ok: true, snapshot_id: built.id, repos: built.repos.length, repositories: built.repos };
   } catch (err) {
     outcome = { ok: false, snapshot_id: null, repos: 0, error: errorMessage(err) };
   }
-  const result: RefreshResult = { event: "snapshot_refresh", snapshot_name: snapshotName, duration_ms: Date.now() - start, ...outcome };
+  const result: RefreshResult = {
+    event: "snapshot_refresh", snapshot_name: snapshotName, commit: commit ?? null, duration_ms: Date.now() - start, ...outcome,
+  };
   if (result.ok) log.info("snapshot refreshed", result);
   else log.error("snapshot refresh failed", result);
   return result;
@@ -43,6 +47,7 @@ if (process.argv[1] && import.meta.url === pathToFileURL(realpathSync(process.ar
   const log = createLog();
   const result = await refreshSnapshot({
     snapshotName,
+    commit: process.env.RENDER_GIT_COMMIT,
     log,
     rebuild: async () =>
       rebuildSnapshot(SandboxClient.fromConfig(await loadRenderConfig(), log), {

src/sandbox/index.ts

@@ -5,7 +5,7 @@ import { SandboxClient } from "./render/client.ts";
 import { loadRenderConfig } from "./render/config.ts";
 import { openRenderSandbox } from "./render/sandbox.ts";
 
-export { SnapshotUnavailableError, type OpenSandbox, type RunInvestigationResult, type Sandbox } from "./interface.ts";
+export { SandboxMisconfiguredError, SnapshotUnavailableError, type OpenSandbox, type RunInvestigationResult, type Sandbox } from "./interface.ts";
 
 /**
  * Loads sandbox config and returns a way to open investigation sandboxes. Selects by `SANDBOX_PROVIDER`

src/sandbox/interface.ts

@@ -70,6 +70,11 @@ export class SnapshotUnavailableError extends Error {
   }
 }
 
+/** Thrown when the provider rejects the Sandbox settings themselves, such as the egress policy. Retrying won't help. */
+export class SandboxMisconfiguredError extends Error {
+  override name = "SandboxMisconfiguredError";
+}
+
 const STDERR_TAIL_CHARS = 4096;
 
 export const appendStderrTail = (tail: string, chunk: string): string => (tail + chunk).slice(-STDERR_TAIL_CHARS);

src/sandbox/render/sandbox.ts

@@ -5,6 +5,7 @@ import { z } from "zod";
 import { RepoManifest } from "../../manifest.ts";
 import { errorMessage } from "../../util.ts";
 import {
+  SandboxMisconfiguredError,
   SnapshotUnavailableError,
   appendStderrTail,
   parseRunnerEvents,
@@ -110,7 +111,7 @@ export async function openRenderSandbox(client: SandboxClient, snapshotName: str
     }
     if (signal.aborted) throw timedOut(err);
     if (policy.type === "allow-list" && err instanceof ClientError && [400, 422].includes(err.statusCode)) {
-      throw new Error(
+      throw new SandboxMisconfiguredError(
         `Render rejected HTTPS egress allow-list [${policyRules(policy).join(", ")}]: ${err.message}`,
         { cause: err },
       );

src/sandbox/render/snapshot.ts

@@ -19,11 +19,11 @@ export type BuiltSnapshot = { id: string; sandboxGroupId: string };
 export async function rebuildSnapshot(
   client: SandboxClient,
   opts: { profile: RepoProfile; snapshotName: string; keep: number },
-): Promise<{ id: string; repos: number }> {
+): Promise<{ id: string; repos: string[] }> {
   const heads = await resolveHeads(opts.profile, client.cfg.hostCredentials);
   const built = await buildSnapshot(client, { profile: opts.profile, heads, snapshotName: opts.snapshotName });
   await pruneSnapshots(client, built, opts.snapshotName, opts.keep);
-  return { id: built.id, repos: heads.length };
+  return { id: built.id, repos: heads.map((h) => h.repository) };
 }
 
 /** Resolves the current default-branch (or configured branch) commit of each repo with `git ls-remote`. */

src/tasks.ts

@@ -10,7 +10,7 @@ import {
 } from "./investigation.ts";
 import type { RepoManifest } from "./manifest.ts";
 import { RUNNER_EXIT, RunnerInput, type RunnerEvent, type Usage } from "./runner/protocol.ts";
-import { SnapshotUnavailableError, type OpenSandbox, type RunInvestigationResult, type Sandbox } from "./sandbox/index.ts";
+import { SandboxMisconfiguredError, SnapshotUnavailableError, type OpenSandbox, type RunInvestigationResult, type Sandbox } from "./sandbox/index.ts";
 import { getBuiltinModel } from "@earendil-works/pi-ai/providers/all";
 import { signCallback } from "./callback.ts";
 import type { Log } from "./log.ts";
@@ -294,6 +294,10 @@ export async function investigateTask(
     sandbox = await openSandbox(config.snapshotName, { timeoutMs: config.sandboxProvisionTimeoutMs });
   } catch (err) {
     if (err instanceof SnapshotUnavailableError) return { kind: "failed", code: err.notReady ? "snapshot_not_ready" : "no_snapshot" };
+    if (err instanceof SandboxMisconfiguredError) {
+      log.warn("sandbox settings rejected", { event: "workflow_misconfigured", err });
+      return { kind: "failed", code: "misconfigured" };
+    }
     throw err;
   }
   log.info("sandbox ready", { sandbox_id: sandbox.id, ...sandbox.info });