[bot] Instrument NVIDIA NeMo Guardrails (149,615 weekly downloads)

[braintrust-bot]

Summary

nemoguardrails (NVIDIA NeMo Guardrails) is a toolkit for adding programmable guardrails to LLM-based conversational applications. Its primary runtime object, LLMRails, exposes generate() / generate_async() / stream_async() as the execution entrypoint apps call instead of invoking a provider client directly — the guardrail "rails" pipeline (input rails, dialog rails, output rails, retrieval rails) sits between the caller and the underlying LLM. This repo has no instrumentation for it.

Why this is not already covered by existing integrations

Since v0.22 (current latest: v0.24.0, released 2026-08-26), NeMo Guardrails' default DefaultFramework LLM-calling path talks directly to OpenAI-compatible endpoints over raw httpx, bypassing both the openai Python client and LangChain entirely. This means:

• Braintrust's existing openai wrapper instrumentation does not see these calls (no openai.OpenAI/AsyncOpenAI client is used on this path).
• Braintrust's existing LangChain instrumentation only applies if a user explicitly opts in via NEMOGUARDRAILS_LLM_FRAMEWORK=langchain config, which routes calls through a LangChainLLMAdapter wrapping BaseLLM/BaseChatModel. This is not the default and cannot be assumed.

So for the default/most common configuration, the entire guardrails-wrapped generation call is currently invisible to Braintrust tracing. A repo-wide grep for nemoguardrails, nemo_guardrails, nemo-guardrails under py/ returns zero matches.

What needs to be instrumented

• LLMRails.generate() / LLMRails.generate_async() — the core guardrailed-generation entrypoint (accepts messages or a raw prompt, returns the final response after rails processing).
• LLMRails.stream_async() — streaming variant.

Weekly downloads

Weekly downloads: 149,615 (as of 2026-09-14; https://pypistats.org/api/packages/nemoguardrails/recent). Independently checked twice against the same source.

Braintrust docs status

not_found — checked https://www.braintrust.dev/docs/guides/tracing/integrations, no mention of NeMo Guardrails, NVIDIA, or nemoguardrails.

Upstream sources

• PyPI: https://pypi.org/project/nemoguardrails/ (v0.24.0, released 2026-08-26)
• GitHub: https://github.com/NVIDIA-NeMo/Guardrails
• Custom LLM framework / default execution path docs: https://docs.nvidia.com/nemo/guardrails/configure-guardrails/custom-initialization/custom-llm-framework (documents the default DefaultFramework calling OpenAI-compatible endpoints directly, and the opt-in LangChain adapter path)

Local repo files inspected

• py/src/braintrust/integrations/ — no nemoguardrails/ directory
• py/pyproject.toml [tool.braintrust.matrix] — no nemoguardrails entry
• py/noxfile.py — no test_nemoguardrails session
• Repo-wide case-insensitive grep for nemoguardrails, nemo_guardrails, nemo-guardrails under py/ — zero matches