Skip to content

Structured image builds, images.json, and the dual bidding gateway - #16

Open
KA-ROM wants to merge 8 commits into
mainfrom
feat/buildernet-node
Open

KA-ROM wants to merge 8 commits into
mainfrom
feat/buildernet-node

Conversation

@KA-ROM

@KA-ROM KA-ROM commented Sep 21, 2026

Copy link
Copy Markdown

Structured image builds, images.json, and the dual bidding gateway

Three changes to the buildernet-node recipe and the image machinery behind
it. The rendered manifests are pinned by a golden test, so the diff there is
the review.

Two bidding gateways, as staging runs them

A production builder ships blocks to two gateways, one collocated on loopback
and one remote. The recipe renders both: gateway-local inside the el-1 pod
and bidding-gateway-1 beside it, with the ports, shared auth token and
health polling taken from the staging inventory
(aws_rbuilder_staging_use2), plus the builder identifiers a gateway checks a
block against before it seals.

An image build is described in fields, not a shell line

ImageBuildSpec now carries dockerfile or assetDockerfile, target,
buildArgs, secrets, assets and the release version behind the pinned
commit. The docker or podman command is derived from those fields, and cmd
stays as an escape hatch for anything they cannot express.

build writes images.json

Next to the manifests, manifests/<recipe>/images.json lists every image the
recipe references, keyed by the tag the manifests use, with the build that
produces it:

"localhost:30500/decker-flowproxy:22aa1d57…-b40f8e0a4a19": {
  "repo": "https://github.com/flashbots/rbuilder-prism",
  "ref": "22aa1d5764076b21f86515f73a48a9978ffb1376",
  "version": "v1.16.0",
  "assetDockerfile": "flowproxy.Dockerfile",
  "secrets": ["gh_token"],
  "cmd": "$ENGINE build --build-context src=. …"
}

The _assets files it names are copied out beside it. That is what lets an
external builder, CI or an in-cluster buildkit, build exactly the tags the
rendered manifests reference without parsing a command string or keeping its
own copy of the pins. ALP now does exactly this, which is how a pin drift bug
class disappears.

One fix that came out of it: the builder and gateway images build from a
private repo with private cargo dependencies, and their specs did not declare
the gh_token secret. Nothing noticed because the images were already built.

The builder container is three files

rbuilder-operator-reth.ts keeps the container; its config template and
production's eight ordering builders move into a companion directory. The
prototype loader only reads files, so a directory beside the module is free.

Tests

20 new deno tests: the derived build command, the images.json payload, the
TOML override rules, the genesis predeploys, the recipe's option surface and
the generic container image override. The k8s render of buildernet-node is
pinned golden, images.json included, so a changed pin has to be an explicit
golden update.

Caret-ranged npm deps resolved at run time to @lodestar/types 1.47+ with @chainsafe/ssz 1.8, which are mutually broken (List limit must be a positive integer); every render failed. deno.json also keeps lint's no-import-prefix off: inline npm:/jsr: specifiers are the repo convention.
Two fixes needed to render a decker stack into a real cluster rather than
a laptop docker daemon.

k8s validates containerPort names pod-wide and Services reference them by
name, but decker containers reuse "http"/"metrics" across containers in one
pod, so the apply was rejected. Assign pod-unique <=15-char port names and
dedupe Service ports on the numeric port instead of erroring on duplicate
names.

Image handling assumed a local daemon: ImageBuildSpec tags have no registry
host, so kubelet tried docker.io and got ImagePullBackOff. Add
DECKER_IMAGE_REGISTRY to prefix built tags and DECKER_IMAGE_MODE=pull to
make ensureImages a no-op so an external builder can supply the images,
plus imagePullPolicy: IfNotPresent for registry-less tags. `build --pods
<renderer>` lets a recipe be rendered for k8s without editing the recipe,
and DECKER_RELAY_HOST lets relay-warmup reach the relay by Service name
instead of localhost.
Any container accepts config: { image } from its recipe or an --opt, applied in the one path all renderers build containers through. Prototype defaults stay where they are; version bumps no longer edit container files.
haproxy -> flowproxy -> rbuilder-operator-reth -> bidding-gateway -> relay, pinned to what flashbots-images trunk/buildernet runs (rbuilder-prism v1.14.0, flowproxy-private v2.13.0, lighthouse v8.2.0); configs transcribed from the production mustache templates.

Factory options: rbuilder.<key>=<raw toml> overrides one line of the production-shaped rbuilder.toml; withdrawals=none (BLS creds); triePadding=<n> (warmup pre-creates accounts). ImageBuildSpec.name/variant, ensureClone for tags/SHAs, lighthouse image/feeRecipient overrides.
…tracts

Via artifacts genesisAccounts. The gateway seal pays the proposer through the mainnet PaymentForwarder and folds only builder+proposer balances into the shipped tries; without the contract every sealed block failed 'mismatched block state root'. The EIP-4788/2935/7002/7251 contracts make the per-block system calls hit real code, as on mainnet.
…m; relay pinned to a SHA

flowproxy moved into rbuilder-prism (crates/flowproxy); build it at the node's ref with the two _assets twins re-based on docker/Dockerfile.flowproxy (upstream: rbuilder-prism #311 --chain-id, #312 arch-agnostic Dockerfile). Bump everything to v1.16.0 (production moved the same day). Pin mev-boost-relay + housekeeper to a commit via config.ref: the container default 'main' is a mutable tag.
A production builder ships blocks to two gateways: one collocated on loopback
and one remote. The recipe now renders both - gateway-local in the el-1 pod
and bidding-gateway-1 beside it - with the ports, auth token and health
polling from the staging inventory, and the builder identifiers the gateway
checks a block against before sealing.
An ImageBuildSpec now carries dockerfile, target, build args, secrets and
decker-owned assets instead of a shell line, and the docker/podman command is
derived from them. build writes manifests/<recipe>/images.json - every image
the recipe references, with the build that produces it and the release name
behind the pinned commit - plus the _assets it names. That is what lets an
external builder (CI, or an in-cluster buildkit) build exactly the tags the
manifests reference without keeping its own copy of the pins.

rbuilder-operator-reth splits into the container, its config template and
production's ordering builders. 20 tests cover the derived build command, the
images.json payload, the TOML override rules, the genesis predeploys, the
recipe's options and the container image override, and the k8s render is
pinned golden, images.json included.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant