[chris-colinsky]

Summary

Part 4 of proposal 0126 (spec v0.119.0). The OpenAI provider now logs one WARNING when a schema-constrained attempt returns content that constrained decoding cannot produce (observability §5.5.7).

• When it fires. The attempt took the "native" path (strict: true), failed structured_output_invalid, finished with finish_reason "stop", carried no stop sequence (declared, or stop in extras, where an empty list counts as none), and returned content that is not a JSON object: malformed JSON, or JSON whose top level is not an object. A JSON object that fails validation does not fire it. Neither does length, content_filter (an OpenAI refusal included), a stop sequence, or any other path.
• What it carries. The event name openarmature.structured_output.schema_not_honored (§7), openarmature.llm.provider, openarmature.llm.model (the bound model, not the one the response reports), openarmature.llm.call_id, which joins it to the call's typed event, and openarmature.llm.attempt_index. Through the log bridge it also gets openarmature.correlation_id, and inside a node body the node span's trace and span ids.
• Advisory. The call still raises StructuredOutputInvalid. Nothing is retried or switched to the fallback because of it.

What changes

• Per wire attempt, independent of observers. The check runs in the provider's per-attempt hook, before the attempt event's dispatch check, so it logs once per attempt with or without an invocation or observers attached. A reask that recovers still leaves the failed attempt's record, and two failing attempts give two records.
• Each attempt's own request. The path and the stop sequences come from the attempt's request as sent, so a per-attempt override that adds a stop sequence suppresses only its own attempt's record.
• One strict parser. The content check and the validation share one JSON loader, so NaN, a number past a float's range, or nesting too deep to parse counts as malformed in both.
• The name is exported as STRUCTURED_OUTPUT_SCHEMA_NOT_HONORED from openarmature.observability.
• Docs: the CHANGELOG, a paragraph under Strict mode on the LLM concepts page, a row in the observability page's event-name table, and the 0126 manifest note, which now also records Describe each LLM request as sent, with its path #347's rule that a request whose connection never opened reports no path.
• Fixture 161 stays deferred: the observability runner lacks the harness support it needs, log-record assertions among them. The unit tests cover its cases 1 to 9, 12 to 16, 18 and 19. Cases 10 and 11 assert the path, which Describe each LLM request as sent, with its path #347's tests cover, and case 17 waits for streaming (proposal 0062).

Verification

• Full suite: 2490 passed. ruff, pyright and mkdocs build --strict are clean.
• Mutation checks, each restored and verified by content: 19 of 19 killed. They drop each arm of the condition, count native_unenforced as native, ignore either stop channel or count an empty list as a stop sequence, parse with plain json.loads, count an array as an object, move the check after the dispatch check, fire once per call, read the base request instead of the attempt's, hardcode the provider, fix the attempt index, drop the call id or the event name, and log at INFO.
• An OTel test drives a failing call through the provider and install_log_bridge, and checks the exported log record's event name field, attributes and correlation id.

[Copilot]

🟡 Changes recommended

The stop-sequence check uses raw configuration instead of the reconciled request value, causing incorrect warning behavior for malformed extras and empty strings; reconciliation logic and regression tests are needed.

1 open issue

• Check reconciled stop value when deciding whether to warn · New

What changed in this PR

Adds an advisory warning when a strict structured-output attempt returns content that is not a JSON object, implementing part 4 of proposal 0126 without changing retry or fallback behavior.

Changes:

• Emits a named warning per qualifying attempt, carrying provider, model, call ID and attempt index.
• Shares strict JSON parsing between validation and warning detection.
• Adds provider and OpenTelemetry tests, exports the diagnostic name, and updates documentation and conformance notes.

File	Description
tests/​unit/​test_observability_otel.py	Verifies warning export, attributes and correlation through the log bridge.
tests/​unit/​test_llm_provider.py	Tests warning conditions, exclusions and per-attempt behavior.
src/​openarmature/​observability/​diagnostics.py	Defines the diagnostic event-name constant.
src/​openarmature/​observability/​__init__.py	Exports the new constant publicly.
src/​openarmature/​llm/​providers/​openai.py	Adds warning detection and emission using shared JSON parsing.
docs/​concepts/​observability.md	Lists the warning in the diagnostic event table.
docs/​concepts/​llms.md	Explains warning conditions and advisory behavior.
conformance.toml	Records implementation progress and deferred fixture coverage.
CHANGELOG.md	Documents the warning and its metadata.

_{Review effort: Balanced}