Skip to content

feat: Bedrock Converse codec and typed routed-call failure contract - #1

Open
c1-squire-dev[bot] wants to merge 3 commits into
mainfrom
paul.querna/bedrock-converse-codec
Open

feat: Bedrock Converse codec and typed routed-call failure contract#1
c1-squire-dev[bot] wants to merge 3 commits into
mainfrom
paul.querna/bedrock-converse-codec

Conversation

@c1-squire-dev

@c1-squire-dev c1-squire-dev Bot commented Aug 12, 2026

Copy link
Copy Markdown

Summary

Two independent public-surface additions staged on this fork branch for review.

Rebased onto current upstream main (5879c10). This branch previously sat on
fb3fc30, 133 commits behind. main on this fork was a stale mirror with no commits of its
own and has been fast-forwarded to the same revision, so this PR is exactly the three commits
below.

Upstream had moved the code these changes touch, so the second and third commits were ported
rather than replayed: routing fallback classification now lives in
libsy-llm-client::run::fallback_reason (it moved out of libsy::core::algorithm), the
escalation classifier lives in libsy/src/algorithms/escalation.rs, and the new
switchyard-runner::failure classifier is an additional dispatch point that did not exist
before. Per-session context-window eviction memory was deleted upstream; see the third commit
for how that affects the overflow case.

Bedrock Converse translation codec

  • add bedrock_converse as a first-class protocol format
  • translate buffered Bedrock Converse requests and responses through Switchyard’s neutral model
  • decode and encode AWS EventStream semantic events incrementally
  • cover tools, reasoning, guardrails, usage, stop reasons, malformed events, and cross-format round trips

Typed routed-call failure contract

Lets a routing host classify its own call failures instead of having libsy re-derive
fallback policy from an HTTP status.

  • add RoutedCallFailure, RoutingDisposition, and the non-exhaustive
    RoutedCallFailureClass with immutable stable_tag() values to switchyard-protocol,
    plus LlmClientError::RoutedCall
  • add bounded provider-exhaustion evidence (RoutedFailureCount,
    ProviderTargetsExhaustedSummary). Validating constructors are the only way to build
    these, so a value always satisfies the contract: candidates bounded to 16, unique
    nonzero class entries that partition attempts and circuit-open bypasses, no nested
    exhaustion, and retry advice capped at 300000 ms
  • libsy fallback now follows the host's explicit RoutingDisposition for a RoutedCall.
    The existing 403/408/429/5xx inference is untouched for the transport- and
    status-shaped variants, so current clients keep their behavior
  • map RoutedCall at every semantic dispatch point: libsy fallback classification, judge
    bounded reasons, client observability labels (reusing the immutable stable tag), and
    switchyard-server status/body mapping with a rounded-up Retry-After when the host
    supplied truthful advice

The types are provider-neutral by construction: bounded class tags, an optional bounded
provider status, and counts only — no provider body, message, or target name.

Follow-up commit fix(routing): honor routed-call classes at every fall-through and status seam closes three gaps found in review of the commit above:

  • the staged-escalation fall-through matched only the legacy ContextWindowExceeded, so a
    host returning RoutedCall terminated the run instead of escalating to the capable tier.
    It now also accepts an advanceable ContextWindow or TargetIncompatible, and still
    treats Stop as terminal
  • ProviderTargetsExhausted collapsed to 503 with its bounded summary unread. The status
    now resolves from the partition in precedence order — exact availability evidence or a
    circuit bypass, then host or provider timeouts, then the homogeneous rate-limit and
    request-shape cases, then a plain routing failure — so the 429, 504, and 422 codes are
    reachable and 503 stays truthful. No provider status is consulted
  • an exhaustion whose every real failure was an overflow reported target unavailability,
    which is exactly the conflation this contract exists to remove. It is now reported as a
    context-window condition at both seams that consume the distinction: the candidate hop's
    reason, and the runner's terminal telemetry kind, where it previously fell into Other.
    Bypassed circuits neither establish nor contradict the overflow, and an all-bypassed
    exhaustion stays unavailability; mixed classes remain unavailability

ProviderTargetsExhaustedSummary grows real_failures and is_context_window_exhaustion so
the aggregate rules are stated once for all three consumers, and rejects an over-long
class-entry list before it sizes an allocation. No existing public type, constructor
signature, or client requirement changes.

Note for reviewers: upstream deleted per-session overflow memory, so the repeat-overflow cost
the third bullet originally eliminated is real again on this base for every client, not just
routed-call hosts. This branch keeps the classification truthful but does not re-add the
memory — that reverses an upstream decision and is worth deciding deliberately.

Validation

  • cargo fmt --all --check
  • cargo build --workspace
  • cargo test --workspace -- --test-threads=1
  • cargo clippy --workspace --all-targets -- -D warnings
  • RUSTDOCFLAGS=-D warnings cargo doc --locked --no-deps --package switchyard-libsy --package switchyard-protocol
  • translation conformance suites for buffered and streaming requests/responses

This fork PR is the review staging point before a human submits the change upstream to
NVIDIA-NeMo/Switchyard, per D-TB-103.

@c1-squire-dev
c1-squire-dev Bot force-pushed the paul.querna/bedrock-converse-codec branch from f3d5b0f to 567e914 Compare August 12, 2026 02:21
@c1-squire-dev c1-squire-dev Bot changed the title Add Bedrock Converse translation codec feat: Bedrock Converse codec and typed routed-call failure contract Sep 4, 2026
pquerna and others added 3 commits September 4, 2026 23:10
Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
Signed-off-by: Paul Querna <paul.querna@conductorone.com>
…tus seam

The typed routed-call contract landed with three consumers still reading the
legacy shape, so a host that classifies its own failures lost behavior the
status-shaped path had.

- staged escalation: the efficient-tier fall-through matched only
  `ContextWindowExceeded`, so a host returning `RoutedCall` terminated the run
  instead of escalating. It now also accepts an advanceable `ContextWindow` or
  `TargetIncompatible`, and still honors `Stop` as terminal.
- exhaustion status: `ProviderTargetsExhausted` collapsed to 503 with its
  bounded summary unread. It now resolves from the partition in precedence
  order -- exact availability evidence or a circuit bypass, then host or
  provider timeouts, then homogeneous rate-limit and request-shape cases, then
  a plain routing failure -- so the 429, 504, and 422 public codes are
  reachable and 503 stays truthful.
- overflow exhaustion: an exhaustion whose every real failure was an overflow
  reported target unavailability, which is the conflation this contract exists
  to remove. It is now reported as a context-window condition at both seams
  that consume the distinction: the candidate hop's reason, and the runner's
  terminal telemetry kind. Bypassed circuits neither establish nor contradict
  the overflow, and an all-bypassed exhaustion stays unavailability. Mixed
  classes remain unavailability.

`ProviderTargetsExhaustedSummary` grows `real_failures` and
`is_context_window_exhaustion` so the aggregate rules are stated once, and
rejects an over-long class-entry list before it sizes an allocation.

Behavioral tests cover each seam, including the five aggregate-precedence
combinations. No existing public type, constructor signature, or client
requirement changes.

Staged on the D-TB-103 immutable review fork ahead of upstream submission.

Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
Signed-off-by: Paul Querna <paul.querna@conductorone.com>
@c1-squire-dev
c1-squire-dev Bot force-pushed the paul.querna/bedrock-converse-codec branch from 0621ca8 to 3036a85 Compare September 5, 2026 00:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant