Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,8 @@ strict output schemas.
## Cross-cutting changes

- Configuration key: update `FactoryConfig` in `factory.config.ts` and
`docs/configuration.md`.
`docs/configuration.md`. Check what the type system can't in
`validateConfig` (`app/config/index.ts`).
- Environment variable: update `.env.example`, `SERVICE_ENV` and, for a
secret, `ENV_GROUPS` in `scripts/lib/requirements.ts`, and
`docs/configuration.md`. Secrets belong in an env group, not a
Expand All @@ -99,8 +100,9 @@ strict output schemas.
- App permission, event, or label: update
`scripts/lib/requirements.ts`, setup behavior, and doctor tests.
- Agent harness: declare the env var of its key as the adapter's
`credential`; setup and `doctor` derive the key list. It must give the
model only the factory's tools; add it to `HARNESSES` in
`credential` and the hosts it calls as its `egress`; setup and `doctor`
derive the key list, and Sandbox allow-lists include every harness's
hosts. It must give the model only the factory's tools; add it to `HARNESSES` in
`tests/helpers/runner.ts` if it can run the conformance suite.
- `sandbox/`: refresh every repository snapshot after deployment. Changes to
`app/runner/`, `app/harness/`, or agents reach Sandboxes with the Workflows
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,11 @@ tests, authority rules, deploys, and snapshot refreshes.
submodules, oversized changes, and branches with human commits.
- GitHub must reproduce the candidate tree SHA before the factory publishes
it.
- Sandboxes reach only an HTTPS allow-list: GitHub, the agents' model APIs,
and the hosts each repository's install and verify commands need.

Sandboxes use allow-all egress, and the model key exists in runner memory
during a call. Read the [trust boundaries](docs/architecture.md#trust-boundaries)
Allowed hosts can still carry data out, and the model key exists in runner
memory during a call. Read the [trust boundaries](docs/architecture.md#trust-boundaries)
and [residual risks](docs/architecture.md#residual-risks) before using private
code or untrusted pull requests.

Expand Down
23 changes: 22 additions & 1 deletion app/config/index.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,28 @@
import config, { type RepoConfig } from "../../factory.config.js";
import config, { type FactoryConfig, type RepoConfig } from "../../factory.config.js";

export { config, type RepoConfig };

/** A lowercase hostname with at least one dot, as the Sandbox egress rules take it; `*.` only as the leftmost label. */
const EGRESS_HOST = /^(\*\.)?([a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$/;

/** Throws naming the first host that isn't a bare hostname: no scheme, port, path, or uppercase. */
export function assertEgressHosts(hosts: readonly string[], where: string): void {
for (const host of hosts) {
if (!EGRESS_HOST.test(host)) {
throw new Error(`${where}: egress host ${JSON.stringify(host)} must be a lowercase hostname such as registry.npmjs.org or *.example.com`);
}
}
}

/** Checks what the type system can't. Runs when this module loads, so an invalid config stops every service at startup. */
export function validateConfig(c: FactoryConfig): void {
for (const [fullName, repo] of Object.entries(c.repos)) {
if (repo.egress !== undefined && repo.egress !== "allow-all") assertEgressHosts(repo.egress, `factory.config.ts repos["${fullName}"]`);
}
}

validateConfig(config);

/** One repository a run acts on, resolved from the webhook that started it. */
export interface RepoContext {
owner: string;
Expand Down
6 changes: 6 additions & 0 deletions app/harness/pi.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@ export interface PiOptions {
models?: () => MutableModels;
/** The env var holding the key. Default: the provider's, from PI_PROVIDER_KEY_ENV; required for other providers. */
credential?: string;
/**
* The hosts model requests go to. Default: the host of `model.baseUrl`. Set it when `models` resolves the model
* to a different base URL, such as a proxy.
*/
egress?: string[];
}

/** Runs an agent on Pi (`@earendil-works/pi-agent-core`) with the factory's tools. */
Expand All @@ -37,6 +42,7 @@ export function pi(opts: PiOptions): Harness {
name: "pi",
label: `${provider}/${id}`,
credential,
egress: opts.egress ?? [new URL(opts.model.baseUrl).hostname],
driver: piDriver({ model: opts.model, thinkingLevel, models: opts.models }),
};
}
5 changes: 5 additions & 0 deletions app/harness/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,9 @@ export interface Harness extends HarnessRuntime {
* for each run; setup and doctor manage it.
*/
readonly credential: string;
/**
* The HTTPS hosts the harness calls from the sandbox, such as its model provider's API. Sandboxes block every
* destination not on their allow-list, so a host missing here makes every model request fail.
*/
readonly egress: readonly string[];
}
42 changes: 42 additions & 0 deletions app/sandbox/egress.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import type { SandboxNetworkPolicy } from "@renderinc/sdk/experimental";
import { AGENTS } from "../agents/registry.js";
import { assertEgressHosts, type RepoContext } from "../config/index.js";

/** Clone and fetch reach GitHub over HTTPS. */
const GITHUB_HOST = "github.com";
/** install-tools.sh installs tools the base image lacks from Debian's mirror, switched to HTTPS. */
const DEBIAN_MIRROR = "deb.debian.org";

/** Every registered agent's model hosts: a sandbox opens before its task picks agents, and one sandbox can run several. */
const MODEL_HOSTS = AGENTS.flatMap((agent) => agent.harness.egress);

/**
* The policy a task sandbox is created with. Render fixes it at creation: a restored snapshot doesn't carry one,
* and a running sandbox's policy can't change. Under `allow-list` only HTTPS to these hosts gets out.
*/
export function taskEgress(repo: RepoContext): SandboxNetworkPolicy {
return egressPolicy(repo, [GITHUB_HOST, ...MODEL_HOSTS]);
}

/** The snapshot builder's policy: it never calls a model, but it installs base-image tools. */
export function builderEgress(repo: RepoContext): SandboxNetworkPolicy {
return egressPolicy(repo, [GITHUB_HOST, DEBIAN_MIRROR]);
}

/** Appended to install failures: a blocked host fails as a reset connection, which reads like a network flake. */
export function egressHint(repo: RepoContext, policy: SandboxNetworkPolicy): string {
if (policy.type !== "allow-list") return "";
return (
`\nSandbox egress is limited to ${policy.rules.map((rule) => rule.domain).join(", ")}. ` +
`If install needs another host, add it to egress for "${repo.fullName}" in factory.config.ts.`
);
}

function egressPolicy(repo: RepoContext, base: readonly string[]): SandboxNetworkPolicy {
const extra = repo.config.egress ?? [];
if (extra === "allow-all") return { type: "allow-all" };
// The API rejects a domain listed twice.
const domains = [...new Set([...base, ...extra])];
assertEgressHosts(domains, `sandbox egress for ${repo.fullName}`);
return { type: "allow-list", rules: domains.map((domain) => ({ domain })) };
}
13 changes: 8 additions & 5 deletions app/sandbox/sandbox.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import { randomUUID } from "node:crypto";
import { SandboxSnapshotNotFoundError, SandboxSnapshotNotReadyError } from "@renderinc/sdk";
import type { SandboxNetworkPolicy } from "@renderinc/sdk/experimental";
import { z } from "zod";
import { config, type RepoContext } from "../config/index.js";
import { type SandboxClient, sandboxClient } from "./client.js";
import { egressHint, taskEgress } from "./egress.js";
import { errorMessage, log, SAFE_REF } from "./shell.js";
import {
asRunner,
Expand All @@ -19,7 +21,7 @@ import {
snapshotName,
} from "./snapshot.js";
import { APP_DIR, type OpenedSandbox, REPO_DIR, RUN_AS, type Sandbox, type SandboxRef, SCRATCH_DIR, type SnapshotManifest } from "./types.js";
import { ALLOW_ALL_EGRESS, verifySandbox } from "./verify.js";
import { verifySandbox } from "./verify.js";

const SnapshotManifestSchema = z.object({
repo: z.string(),
Expand Down Expand Up @@ -66,7 +68,8 @@ export interface OpenSandboxDeps extends SnapshotDeps {
export async function openSandbox(repo: RepoContext, ref: SandboxRef, deps: OpenSandboxDeps = {}): Promise<OpenedSandbox> {
checkRef(ref);
const client = deps.client ?? sandboxClient();
const id = await createFromSnapshot(client, repo, deps);
const egress = taskEgress(repo);
const id = await createFromSnapshot(client, repo, egress, deps);
try {
await client.waitRunning(id);
await ensureRunnerPayload(client, id);
Expand All @@ -82,7 +85,7 @@ export async function openSandbox(repo: RepoContext, ref: SandboxRef, deps: Open
const lockfileHash = await computeLockfileHash(client, id);
if (lockfileHash !== manifest.lockfileHash || install !== manifest.install) {
log("info", "reinstalling dependencies", { sandbox: id, repo: repo.fullName, headSha });
await runInstall(client, id, install);
await runInstall(client, id, install, egressHint(repo, egress));
}
}
return { ...connectSandbox(id, client), headSha, manifest };
Expand Down Expand Up @@ -120,14 +123,14 @@ function isSnapshotUnavailable(err: unknown): boolean {
return err instanceof SandboxSnapshotNotFoundError || err instanceof SandboxSnapshotNotReadyError;
}

async function createFromSnapshot(client: SandboxClient, repo: RepoContext, deps: OpenSandboxDeps): Promise<string> {
async function createFromSnapshot(client: SandboxClient, repo: RepoContext, egress: SandboxNetworkPolicy, deps: OpenSandboxDeps): Promise<string> {
const name = snapshotName(repo);
const create = async () =>
(
await client.call(() =>
client.api.create({
snapshotName: name,
networkPolicy: ALLOW_ALL_EGRESS,
networkPolicy: egress,
timeoutSeconds: config.sandbox.timeoutSeconds,
}),
)
Expand Down
14 changes: 8 additions & 6 deletions app/sandbox/snapshot.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@ import type { RepoContext } from "../config/index.js";
import { githubToken } from "../github/auth.js";
import { bundleRunner } from "../runner/bundle.js";
import { type SandboxApi, type SandboxClient, TRANSIENT } from "./client.js";
import { builderEgress, egressHint } from "./egress.js";
import { errorMessage, log, SAFE_REF, shq } from "./shell.js";
import { APP_DIR, REPO_DIR, RUN_AS, SANDBOX_ROOT, SCRATCH_DIR, type SnapshotManifest } from "./types.js";
import { ALLOW_ALL_EGRESS, verifySandbox } from "./verify.js";
import { verifySandbox } from "./verify.js";

/** The one named snapshot per repository: `factory-<owner>-<name>`, restricted to [A-Za-z0-9_.-]. */
export function snapshotName(repo: Pick<RepoContext, "owner" | "name">): string {
Expand Down Expand Up @@ -78,7 +79,7 @@ export function credentialFile(token: string): string {

// ── Build ───────────────────────────────────────────────────────────────
//
// Each step is an exec or upload in an allow-all builder sandbox, ordered by trust:
// Each step is an exec or upload in a builder sandbox limited to builderEgress, ordered by trust:
//
// 1. Upload the scripts and runner payload; install tools; create the runner user; self-test the runner
// 2. Upload a read-only token, clone at `sha`, scrub the token and verify none remains
Expand All @@ -97,7 +98,8 @@ export async function buildSnapshot(client: SandboxClient, repo: RepoContext, sh
const started = Date.now();
log("info", "snapshot build started", { repo: repo.fullName, sha, snapshot: name });

const { id } = await client.call(() => client.api.create({ timeoutSeconds: BUILDER_TIMEOUT_SECONDS, networkPolicy: ALLOW_ALL_EGRESS }));
const egress = builderEgress(repo);
const { id } = await client.call(() => client.api.create({ timeoutSeconds: BUILDER_TIMEOUT_SECONDS, networkPolicy: egress }));
try {
await client.waitRunning(id);

Expand Down Expand Up @@ -145,7 +147,7 @@ export async function buildSnapshot(client: SandboxClient, repo: RepoContext, sh
await client.check(id, `chown -R ${RUN_AS}: ${REPO_DIR}`, prepare);
const lockfileHash = await computeLockfileHash(client, id);
const install = repo.config.install?.trim() || null;
if (install) await runInstall(client, id, install);
if (install) await runInstall(client, id, install, egressHint(repo, egress));

// ── 4. Manifest, lockdown, verify, snapshot ──────────────────────
const manifest: SnapshotManifest = {
Expand Down Expand Up @@ -222,10 +224,10 @@ export async function computeLockfileHash(client: SandboxClient, id: string): Pr
* Runs the repository's install command as the runner user, then fails if it changed any tracked file: the
* checkout is what agents edit and publish from, so a dirty tree would leak into every PR.
*/
export async function runInstall(client: SandboxClient, id: string, install: string): Promise<void> {
export async function runInstall(client: SandboxClient, id: string, install: string, failureHint = ""): Promise<void> {
const res = await client.exec(id, asRunner("bash", "-c", install), { timeoutMs: client.t.prepareTimeoutMs });
if (res.exitCode !== 0) {
throw new Error(`install (${install}) exited ${res.exitCode}: ${(res.stderr || res.stdout).trim().slice(-2000)}`);
throw new Error(`install (${install}) exited ${res.exitCode}: ${(res.stderr || res.stdout).trim().slice(-2000)}${failureHint}`);
}
const dirty = (
await client.check(id, asRunner("git", "status", "--porcelain", "--untracked-files=no"), {
Expand Down
10 changes: 1 addition & 9 deletions app/sandbox/verify.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,8 @@
import { z } from "zod";
import type { SandboxApi, SandboxClient } from "./client.js";
import type { SandboxClient } from "./client.js";
import { errorMessage } from "./shell.js";
import { APP_DIR } from "./types.js";

type NetworkPolicy = NonNullable<NonNullable<Parameters<SandboxApi["create"]>[0]>["networkPolicy"]>;

/**
* Allow-all egress for every sandbox, in the shape the API accepts, `{ type }`. SDK 1.2.0's types still describe
* the older `{ default, allowedDomains }` shape.
*/
export const ALLOW_ALL_EGRESS = { type: "allow-all" } as unknown as NetworkPolicy;

const SandboxInfo = z.object({
runAs: z.string().min(1),
uid: z.number().int(),
Expand Down
22 changes: 19 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,10 +199,18 @@ Clone and fetch credentials are read-only and temporary. Runtime files are
root-owned and read-only. Agents and repository commands run as an
unprivileged user.

Every Sandbox is created with an HTTPS egress allow-list built by the
Workflows host from trusted configuration. Task Sandboxes reach `github.com`,
the agents' model APIs, and the repository's configured `egress` hosts. The
snapshot builder reaches `github.com`, Debian's package mirror, and the
repository's hosts. Render enforces the list outside the Sandbox and fixes it
at creation, so nothing inside the Sandbox can widen it. See
[Sandbox egress](configuration.md#sandbox-egress).

The model key enters the Sandbox during an agent call. The runner receives it
through a root-only path and stdin, removes provider keys from tool
environments, and runs under a non-dumpable Node binary. These controls reduce
exposure; they do not make untrusted code safe.
environments, and runs under a non-dumpable Node binary. These controls and
the allow-list reduce exposure; they do not make untrusted code safe.

## Security invariants

Expand All @@ -220,12 +228,20 @@ Changes must preserve these properties:
9. Implement the specification as it existed when approved.
10. Give every Sandbox a provider timeout and `finally` cleanup. Record a
GitHub task's Sandbox so cancellation can terminate it.
11. Create every Sandbox with an egress allow-list built from trusted
configuration. Only a repository's configured `egress: "allow-all"` lifts it.

Tests pin these invariants and the tool authority of every built-in agent.

## Residual risks

- Sandboxes use allow-all egress.
- Allowed hosts remain exfiltration paths. Code in a Sandbox can send data to
an attacker's own GitHub account or model API account through `github.com`
or the model API, and to anything a wildcard rule admits. Task Sandboxes
keep `github.com` for their whole lifetime because the policy can't change
after the fetch.
- DNS resolution is not restricted by the allow-list.
- A repository configured with `egress: "allow-all"` has unrestricted egress.
- The model key exists in runner memory during a call.
- Prompt injection can influence specifications, code, and reviews.
- Anyone with `minimumPermission` can start model spend.
Expand Down
45 changes: 45 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,13 @@ export default defineFactoryConfig({
baseBranch: "main",
install: "npm ci",
verify: ["npm run build", "npm test"],
egress: ["registry.npmjs.org"],
},
"acme/api": {
baseBranch: "main",
install: "go mod download",
verify: ["go test ./..."],
egress: ["proxy.golang.org", "sum.golang.org"],
},
},
labels: {
Expand Down Expand Up @@ -52,11 +54,48 @@ from any other repository, even if the App is installed there.
| `baseBranch` | Yes | Branch used for Triage, implementation, and snapshots |
| `install` | No | Dependency command run after clone credentials are removed |
| `verify` | Yes | Commands required before publishing and during PR review |
| `egress` | No | Extra HTTPS hosts this repository's Sandboxes may reach, or `"allow-all"` |

Use deterministic install commands such as `npm ci`. Snapshot creation fails
if installation changes a tracked file. Use `verify: []` only when you
intentionally want no command-based verification.

### Sandbox egress

Sandboxes reach the network through an HTTPS allow-list that Render enforces.
Every other destination fails as a reset connection. The factory builds each
list:

| Sandbox | Allowed hosts |
|---|---|
| Task Sandbox | `github.com`, every agent harness's model API, and the repository's `egress` |
| Snapshot builder | `github.com`, `deb.debian.org`, and the repository's `egress` |

List in `egress` every host the repository's `install` and `verify` commands
download from. Task Sandboxes need the list too: they reinstall when the
lockfiles change. Common hosts:

| Ecosystem | Hosts |
|---|---|
| npm | `registry.npmjs.org` |
| Go modules | `proxy.golang.org`, `sum.golang.org` |
| Go toolchain download | `go.dev`, `dl.google.com` |
| PyPI | `pypi.org`, `files.pythonhosted.org` |

A rule matches one exact hostname. Use `*.example.com` to admit its
subdomains; a wildcard may only be the leftmost label. Only HTTPS is admitted.
An invalid host stops both services at startup. An install that fails under
the allow-list names the allowed hosts in its error.

Set `egress: "allow-all"` to lift the restriction for one repository. Prefer
listing hosts: the allow-list is what stops dependency scripts and
prompt-injected agents from sending code or the model key to arbitrary
destinations.

Render fixes a Sandbox's policy when it is created, so an `egress` change
applies to the next Sandbox after a Workflows deploy. No snapshot refresh is
needed.

After adding a repository:

1. commit and deploy the configuration;
Expand Down Expand Up @@ -124,6 +163,12 @@ needs no other configuration change. The `pi` adapter supports Anthropic
3. deploy `factory-workflows`; and
4. run `npm run doctor -- --service workflows`.

Each harness also declares its `egress`: the hosts its model requests go to.
Every task Sandbox's allow-list includes the hosts of all registered agents.
The `pi` adapter uses the host of the model's `baseUrl`, so switching
providers needs no egress change. Pass `egress` to `pi()` when a custom
`models` registry serves the model from another host, such as a proxy.

### Tool support

Every harness gives the model the factory's own tools, so an agent's tools and
Expand Down
5 changes: 4 additions & 1 deletion docs/runbooks/extend.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,10 @@ driver that only translates the library.
configuration and returns a `Harness` (`app/harness/types.ts`):
- `name` and `label` name the library and model before a run;
- `credential` is the env var on the Workflows service that holds its key;
the host sends its value into the Sandbox; and
the host sends its value into the Sandbox;
- `egress` lists the HTTPS hosts the library calls from the Sandbox, such
as its model API. Sandboxes block every host not on their allow-list, so
derive it from the library's configuration when you can; and
- `driver` is a `Driver` (`app/runner/core/driver.ts`).
2. Write the driver in `app/runner/<name>/`. `open(ctx)` resolves the model
and reports the library version; the session's `run()` converses with the
Expand Down
Loading
Loading