[bot] AWS Bedrock batch inference (CreateModelInvocationJob) is not instrumented at all

[braintrust-bot]

Summary

The aws_bedrock_2_30_0 module only ever wraps the Bedrock data-plane runtime client builders (software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClientBuilder / BedrockRuntimeAsyncClientBuilder), and only tags Converse/ConverseStream operations on that client (tracked separately in #134 for InvokeModel). AWS Bedrock's official batch inference API — submitted through the entirely separate control-plane client software.amazon.awssdk.services.bedrock.BedrockClient (CreateModelInvocationJob / GetModelInvocationJob / ListModelInvocationJobs / StopModelInvocationJob) — has no wrapping mechanism at all. This is a different Maven artifact/class hierarchy (bedrock vs. bedrockruntime) that this module never references, so unlike the Gemini Batches gap (#165, where calls at least reach the wrapper and get mis-tagged), a Bedrock batch inference job call today produces zero span — not even a generic or mis-tagged one.

What is missing

• BraintrustBedrockInterceptor.java line 55: private static final Set<String> INSTRUMENTED_OPERATIONS = Set.of("Converse", "ConverseStream");, enforced at lines 62–66 (beforeExecution returns immediately for any other operation name). This interceptor is only ever attached to a BedrockRuntimeClientBuilder/BedrockRuntimeAsyncClientBuilder in the first place (see next point), so it structurally cannot see batch-inference-job calls regardless of this allowlist.
• BraintrustAWSBedrock.java (lines 23–24, 53–54): both public wrap() overloads are typed to accept only BedrockRuntimeClientBuilder / BedrockRuntimeAsyncClientBuilder. There is no wrap(BedrockClientBuilder) overload, so a manual user has no supported way to instrument the control-plane client that hosts batch inference at all.
• AWSBedrockInstrumentationModule.java (BedrockBuilderAdvice.build(), lines 74–83): the auto-instrumentation instanceof checks are likewise limited to BedrockRuntimeClientBuilder/BedrockRuntimeAsyncClientBuilder. A BedrockClientBuilder (control plane) passing through SdkDefaultClientBuilder.build() falls through both branches untouched.
• InstrumentationSemConv.tagBedrockRequest()/tagBedrockResponse() only understand Converse-shaped JSON (messages, output.message, usage.inputTokens) — even if a batch job call were reached, there's no field extraction for CreateModelInvocationJobRequest/GetModelInvocationJobResponse shapes (jobArn, inputDataConfig/S3 URI, outputDataConfig/S3 URI, status, roleArn).
• No test, cassette, or example anywhere in the repo references CreateModelInvocationJob, GetModelInvocationJob, ModelInvocationJob, or the bedrock (non-bedrockruntime) client package — confirmed via repo-wide search.
• No async submit → poll-status → retrieve-result lifecycle is modeled for this provider, unlike the (also-gapped, but at least structurally reachable) OpenAI ([bot] OpenAI Batch API is not instrumented (mis-tagged as a generic LLM span) #164), Anthropic ([bot] Anthropic Message Batches API is not instrumented (mis-tagged as a generic LLM span) #155), and Google GenAI ([bot] Google GenAI Batches API is not instrumented (mis-tagged as a generic LLM span) #165) batch APIs.

Braintrust docs status: not_found

Fetched https://www.braintrust.dev/docs/integrations/ai-providers/bedrock in full. It documents per-language tracing coverage for Converse/ConverseStream/InvokeModel/InvokeModelWithResponseStream across the TypeScript, Python, Go, and Java SDKs (explicitly noting for Java: "Other Bedrock operations such as InvokeModel and ApplyGuardrail are not traced" — consistent with #134). No mention anywhere on the page, for any language SDK, of batch inference / model invocation jobs / CreateModelInvocationJob. This is a documentation gap across the whole product, not just Java, but the Java SDK itself also has zero code path that could produce a span for this API even if docs existed.

Upstream sources

• AWS Bedrock batch inference user guide: https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference.html and https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference-create.html — describes submitting a batch inference job against an S3 input file and retrieving results from S3 once the job completes.
• API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateModelInvocationJob.html (also GetModelInvocationJob, ListModelInvocationJobs, StopModelInvocationJob).
• AWS SDK for Java v2 control-plane client: software.amazon.awssdk.services.bedrock.BedrockClient#createModelInvocationJob — https://docs.aws.amazon.com/java/api/latest/software/amazon/awssdk/services/bedrock/BedrockClient.html and https://docs.aws.amazon.com/java/api/latest/software/amazon/awssdk/services/bedrock/model/CreateModelInvocationJobRequest.html — this is a distinct Maven artifact (software.amazon.awssdk:bedrock) from the already-instrumented software.amazon.awssdk:bedrockruntime.

Local repo files inspected

• braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java — line 55 (INSTRUMENTED_OPERATIONS), lines 62–66 (allowlist enforcement), imports at lines 31–32 (bedrockruntime.model.ConverseRequest/ConverseStreamRequest only)
• braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrock.java — lines 23–24 and 53–54 (wrap() overload signatures, both bedrockruntime builder types only)
• braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/auto/AWSBedrockInstrumentationModule.java — lines 74–83 (BedrockBuilderAdvice.build() instanceof checks limited to bedrockruntime builders)
• braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — tagBedrockRequest()/tagBedrockResponse() handle only Converse-shaped bodies
• braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java — all tests use ConverseRequest/ConverseStreamRequest; no batch-job test exists
• Repo-wide search for ModelInvocationJob, CreateModelInvocationJob, and the software.amazon.awssdk.services.bedrock. (control-plane) package — zero matches outside this issue
• Existing related/non-duplicate issues reviewed: [bot] AWS Bedrock InvokeModel for embeddings not instrumented #134 (Bedrock InvokeModel embeddings — same module, different operation, not batch), [bot] Google GenAI Batches API is not instrumented (mis-tagged as a generic LLM span) #165 (Google GenAI Batches — different provider), [bot] Anthropic Message Batches API is not instrumented (mis-tagged as a generic LLM span) #155 (Anthropic Message Batches — different provider), [bot] OpenAI Batch API is not instrumented (mis-tagged as a generic LLM span) #164 (OpenAI Batch API — different provider)