Skip to content

Conversation Routing

Defang Agent edited this page Oct 16, 2026 · 5 revisions

Conversation routing and runtime parking

Agentbox should expose conversations, not harness processes. A person sends a message to one entry point; Agentbox selects a logical conversation and then materializes an executor only while that conversation has work to run.

This is a lifecycle and routing boundary, not a new agent harness. Claude Code, Codex and later OpenCode remain responsible for their own transcripts, tool loops, compaction and model APIs. Agentbox stores only enough durable metadata to choose and resume them.

The three identities

Keep these separate:

  • Entry point - WhatsApp first, with other chat or webhook channels later.
  • Logical conversation - a durable Agentbox record with routing metadata and a pointer to a native harness thread or transcript.
  • Runtime activation - a disposable Claude, Codex or OpenCode process that is admitted through the shared capacity policy and resumes that pointer.

A running process is therefore a cache of a conversation. Stopping it must not change the conversation's identity, and activation must not silently create a replacement when the exact target cannot be admitted.

Routing decisions

Mechanism comes before policy. The mechanism must support these outcomes:

  1. resume the selected conversation in its current harness;
  2. start a clean conversation;
  3. start clean with an explicit pointer or capsule from a prior conversation;
  4. later, continue through a different harness or model.

WhatsApp currently supplies an explicit selection (/target) and an automatic new-session fallback. Semantic selection among several conversations is a later policy and must return confidence plus an explainable reason. Low confidence starts clean or asks; it must not contaminate an unrelated transcript.

Model or harness selection is a second decision after conversation selection. Do not combine them into one opaque score: whether history belongs to a request and which executor should run it are different questions.

Cold conversations

A listed, stopped session is the initial cold representation. The native Claude or Codex transcript stays authoritative on disk. An inbound message activates the exact stopped session through the ordinary capacity gate, and the supervisor resumes its native identifier. Admission failure leaves the message queued on that same target.

Parking is demand-driven, not an idle reaper. A completed session remains resident while capacity is free. When a standing-webhook dispatch or exact WhatsApp target activation needs a worker slot and the shared session limit is full, admission may park one eligible resident runtime and immediately retry the same admission. The idle grace controls eligibility; it is not a timer that parks sessions on its own.

Capacity-pressure parking is safe only when quiescence is proven conservatively:

  • the native transcript says the current turn completed;
  • no unresolved assignment lease exists;
  • no live webhook subscription says the session is waiting for external work;
  • no inbound delivery is pending;
  • an activity generation did not change during the idle grace interval.

Unknown, missing or truncated harness state means active. Agentbox already has the Claude transcript predicate used by restart notices (claude_turn_open), and Codex rollout files carry a task_complete event that can provide the parallel predicate. Remote-Control Codex is excluded until one daemon's relationship to app-owned tasks can be proven.

Shell, ephemeral, crashed, webhook-origin and Remote-Control Codex sessions are not candidates. A session with an unresolved lease, a live session webhook subscription or a selected pending WhatsApp delivery also remains resident. The webhook exclusion is intentional until local-channels#60 makes accepted peer delivery durable.

The stable Agentbox session name is the routing identity. Its per-session state records the harness launch ID and the exact current native Claude/Codex ID; Claude rotation records and Codex rollout launch markers keep that mapping current. Parking marks the existing registry entry stopped, records a unique hibernation generation, and removes only its tmux runtime. Activation clears that state through the shared capacity gate and resumes the recorded native conversation. Missing exact-resume evidence fails closed; it never falls back to a new conversation.

Selection and mutation are fenced separately. Transcript and routing evidence is hashed while selecting the oldest eligible runtime, revalidated under the registry lock, and checked again before and after the pane is terminated. If a delivery or explicit activation races with parking, its generation change wins: the stopped marker is cleared and the supervisor resumes the same target. If a different slot becomes free first, the parking transition is rolled back and no runtime is terminated.

This first implementation is tracked by agent-box#858. It supports native Claude and non-Remote-Control Codex conversations. The park operation and inbound delivery race have one invariant: either delivery prevents parking, or it remains durable and reactivates the cold target. Neither outcome may create a second conversation.

State and ownership

Agentbox owns:

  • a stable logical conversation id and entry-point bindings;
  • harness/profile and native transcript/thread pointers;
  • parent or source pointers for a clean continuation;
  • lifecycle state (cold, activating, resident, parking);
  • activity generation, timestamps and routing evidence.

The harness owns transcript contents and execution semantics. The channel owns durable inbound delivery until handoff is acknowledged. This acknowledgement boundary matters: a stopped process is harmless only when accepted messages do not exist solely in RAM.

Sessions remain units of work, not security boundaries. Every runtime under one Linux user shares that user's trust domain, as documented in Users-vs-Sessions.

Related work and what to borrow

  • Conversation disentanglement provides useful features and evaluation methods for deciding which prior thread a message belongs to. It is routing policy, not lifecycle machinery.
  • vLLM session-aware agentic routing contributes session locks, reset/replay semantics and cache-aware scheduling. Its workers are model API backends rather than native interactive harnesses.
  • Pantalk is the closest open-source daemon/driver pattern for connecting chat transports to existing agents. Its deterministic external-thread mapping does not solve several logical conversations inside one WhatsApp self-chat.
  • OpenRath demonstrates session lineage, but adopting its runtime would duplicate the harness layer Agentbox deliberately leaves to Claude, Codex and OpenCode.

Use those projects as design inputs or narrow components. Do not adopt a second agent runtime merely to gain routing.

ACP/acpx compatibility findings (2026-10-15)

Spike #862 tested native Codex 0.162.1 and Claude Code 2.1.296, current Codex ACP 2.2.2 and Claude ACP 0.91.0, acpx 0.19.5, and OpenCode 1.19.0. The issue holds the capability matrix, exact source/package pins, probe evidence, measurements and driver contract. These are versioned observations, not promises about all adapters.

Keep narrow native drivers for the next activation slice. ACP is a viable optional transport; it does not replace Agentbox's identity, admission, durable-ingress or parking policy. acpx can run independently of OpenClaw, but its ordinary CLI reconnect behavior is unsuitable as the authority for an exact cold target.

What worked

Native-created Codex and Claude conversations survived complete process shutdown and resumed through ACP under the original native IDs. Both returned synthetic continuity tokens without transcript conversion. Native resume after ACP writes also worked. acpx explicitly attached to those native IDs and returned the tokens after each owner's complete shutdown. Its local record ID is distinct from the provider/native ID; neither a cwd/name lookup nor an acpx record ID is a replacement for Agentbox's logical conversation ID.

OpenCode's native HTTP server created a session which a fresh built-in ACP process subsequently loaded under the same ID. This was an empty-session lifecycle probe; authenticated OpenCode model continuity remains untested.

No per-conversation daemon had to remain resident. On one 2-CPU aarch64 host, three process-cold samples per path gave these median resident idle process-tree RSS values:

Path Codex Claude
Native 193 MiB 226 MiB
Direct ACP 291 MiB 381 MiB
Through acpx 400 MiB 492 MiB

After teardown, each test tree had zero live processes. RSS is not PSS or host-wide memory savings, and the Claude native/SDK configurations differed. The issue gives peaks, first-output/terminal latency, sample ranges and all limitations. acpx defaults to a 300-second idle owner TTL; setting TTL to zero keeps the owner alive indefinitely. The probe used two seconds and observed natural owner/adapter exit.

Boundaries Agentbox still must enforce

  • Durable ingress: acpx acknowledged a queued synthetic follow-up, then lost it when its owner was killed. Its in-memory queue is not the channel's durable spool. Queue acceptance, native prompt start and terminal completion are separate acknowledgements. A request ID does not make retries exactly once; uncertain delivery requires reconciliation.
  • Exact resume: ordinary acpx CLI reconnect created a fresh provider session after the synthetic provider's saved ID disappeared. The shared runtime API instead returned SESSION_RESUME_REQUIRED. Both its shared runtime and persistent in-process runtime select same-session-only behavior in the inspected source. Any integration must explicitly choose and test that behavior; activation must never fall back to creation.
  • Writer ownership: current Codex rejected a second adapter's resume with thread_active_writer. Two Claude adapters resumed the same native UUID and both completed turns. Agentbox needs its own per-target lease and generation fencing across channels and cooperating clients.
  • Prompt serialization: acpx serialized real follow-ups for both harnesses and allowed a queued turn to complete after cancelling the active turn. Overlapping direct ACP prompts are not a portable queue contract. Claude advertises an extension; the Codex negative probe left the first request unanswered while completing the second.
  • Native permissions: client-side deny policies apply to permission requests that reach the client. The Codex adapter's default Auto review mode approved a disposable write internally. Explicit read-only mode requested client permission and honored denial; Claude honored denial too. Select and validate native modes before submitting work. An end_turn result can describe a denied operation, and CLI exit zero can accompany cancellation.
  • Quiescence: prompt completion does not prove no background work, goal, permission request, lease, subscription or inbound delivery remains. Unknown state stays active. Remote-Control Codex remains excluded from automatic parking until app-owned tasks can be accounted for.

Protocol and driver shape

The current ACP v1 session setup includes capability-gated session/load with replay, session/resume without replay, and session/close. Resume is no longer merely a draft-method assumption. Fork is an RFD, and ACP v2 is draft. Negotiate capabilities rather than inferring them from the harness name.

A minimal driver exposes capabilities, exact activation, serialized submit with structured events, cancellation, quiescence inspection and parking. Creation and fork are separate explicit operations. Persist native ID, state root, cwd, profile and generation alongside the logical conversation. Missing resume, writer conflict or capacity refusal keeps delivery bound to that exact target. ACP load may substitute for resume when supported, but replay must not be delivered as fresh channel output.

Current codex-acp wraps App Server, unlike the deprecated adapter that embedded an older Codex core. It exposes more than basic chat, including fork and adapter-specific extensions, but portable ACP still does not preserve the full native App Server control surface. acpx's acpx/runtime exports are embeddable; its shared owner path lacks some lifecycle/permission hooks available in the in-process runtime and adds a record/event journal. Neither is an authorization to introduce a second authoritative transcript store.

Delegation-only model router comparison (2026-10-15)

A follow-up to spike #862 tested a deliberately simple alternative: use a Claude Code conversation as the semantic router, disable its built-in tools, and expose one allowlisted MCP tool that delegates to an exact existing Codex or Claude native ID. The router saw only two opaque targets and short topic descriptions. The delegate rejected unknown targets and never created a replacement conversation.

The first run started a fresh Sonnet process and router conversation for every prompt. That is a cold/stateless baseline, not the sensible steady-state design. A correction reran the same corpus through one live router process and one persistent router conversation per model, first with Sonnet and then with Haiku. Each model received its own fresh pair of native target conversations, so the continuity test remained independent. The resolved router models were claude-sonnet-5-5 and claude-haiku-5-5.

On a small hand-written corpus, every configuration selected all eight expected outcomes: three Codex targets, three Claude targets, one explicit ambiguity, and one request that tried to make the router bypass delegation and read a file. All six routed turns recovered a continuity token stored only in the selected native conversation. Every turn made exactly one delegate call, including an explicit unrouted call for the two no-match cases. Both persistent routers shut down cleanly with no surviving test process.

This establishes feasibility, not routing quality. The labels were obvious, the catalog had only two targets, samples were not randomized or blind, and the same native histories grew across runs. A production policy still needs an offline replay corpus, confusion costs, calibrated confidence and a safe ask-or-start- clean threshold.

Three samples per native target gave these medians. Time is prompt submission to terminal result; memory is summed process-tree RSS. The old cold/stateless row also includes launch, initialization and shutdown for a fresh router process. The persistent rows keep the router process and its conversation warm between prompts. Every routed path still includes two model turns - one routing decision and one native worker turn - so this is not an adapter-only benchmark.

Path Codex terminal / peak Claude terminal / peak
Native, #862 3.35 s / 193 MiB 1.56 s / 247 MiB
Direct ACP, #862 4.51 s / 291 MiB 2.52 s / 522 MiB
Through acpx, #862 5.57 s / 518 MiB 3.55 s / 766 MiB
Cold/stateless Sonnet router + native 8.43 s / 424 MiB 8.75 s / 459 MiB
Persistent Sonnet router + native 5.81 s / 438 MiB 6.93 s / 481 MiB
Persistent Haiku router + native 6.51 s / 438 MiB 6.22 s / 479 MiB

Across all six routed prompts, persistence reduced Sonnet's median decision time from 2.25 s to 1.36 s and its terminal time from 8.74 s to 6.40 s. Persistent Haiku's corresponding medians were 0.88 s and 6.23 s. Haiku had one 11.42 s terminal outlier, so the small median is not a tail-latency claim. Unrouted warm prompts took a median 3.38 s on Sonnet and 2.16 s on Haiku.

Persistence exchanges launch cost for resident memory. After a routed turn, the warm router's median idle process-tree RSS was 268 MiB for Sonnet and 265 MiB for Haiku. A separate cold calibration measured launch-to-initialization at 0.85 s and a 3.13 s first no-match turn for Sonnet; Haiku measured 0.75 s and 2.16 s respectively. The persistent router history also grew during the run, so its routed peak was slightly higher than the fresh-process baseline.

The correction changes the latency and deployment conclusion: a persistent router is the representative design, and Haiku is viable for this easy corpus. It does not make the router cheaper than direct native execution or remove the extra model call. Its peaks were lower than acpx in this configuration, but configuration and history differences make that a diagnostic observation, not a capacity ratio.

Behavioral probes preserved the mechanism problems from #862:

  • Two router processes selected the same Codex target within 0.56 s. One native resume completed; the other returned thread_active_writer. Two Claude resumes both completed. Semantic routing is not a writer lease.
  • Terminating the router after its delegate call began left the Codex worker orphaned for more than one second. The native transcript later recorded the user message, assistant answer and task_complete, but the delegate never returned a completion receipt. Retrying blindly could duplicate a committed turn; dropping it could lose the reply. Semantic routing is not durable delivery or uncertain-outcome reconciliation.
  • An invalid opaque target was rejected before any worker started. This is the correct fail-closed behavior, but it came from the delegate implementation, not from the model.

The useful boundary is therefore narrower than "use a harness as the router." A tool-limited model can be an optional policy stage that returns a target, confidence and reason after deterministic bindings and active-session locks are checked. The non-model delegate still must own the allowlist, exact native ID, capacity admission, writer lease, durable input reference, generation fence, structured completion and reconciliation. It must expose start-clean as a separate explicit operation, never as resume fallback.

Removing filesystem tools from the router reduces accidental capability and prompt-injection reach; it is not a security boundary. The router, MCP server and workers still ran under one Linux user, and the delegate processes held the native credentials. Use a separate Linux user when the router needs a different trust domain, per Users-vs-Sessions.

Implement #857's exact activation and durable delivery boundary first, using existing admission and native resume mechanisms. Then implement #858 parking with conservative quiescence evidence. Same-harness ACP continuity neither requires nor proves #744's cross-harness transcript conversion.

Tracked implementation

  • agent-box #857 - activate an exact cold WhatsApp target before delivery.
  • agent-box #858 - park quiescent native harness runtimes automatically.
  • agent-box #744 - transfer a conversation between Claude and Codex.
  • local-channels #60 - durable inbound spool so accepted events survive a missing runtime.