Skip to content

docs: correct JSON completion reasoning behavior in configuration #261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
detail-app wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

docs: correct JSON completion reasoning behavior in configuration #261

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Common variables:
- `OPENAI_MODEL` (non-blank gateway chat alias; default `claude-haiku-5-5`)
- `OPENAI_STREAMING_REQUEST_TIMEOUT_SECONDS` (default `90`; bounds the complete SDK call and the visible-output deadline, which shares this same budget)

Non-secret generation policy is owned by `app.llm` in `application.properties`: `temperature`, `reasoning-effort`, `completion-output-token-budget`, `enrichment-output-token-budget`, `reranker-temperature`, `reranker-output-token-budget`, and `configured-provider-backoff-seconds`. The tested default is Claude Haiku 5.5 with `reasoning-effort=low`, which preserved quality while reducing latency and cost in the Java Chat matrix; both temperature settings stay at `1.0` and the gateway owns provider-specific sampling constraints. The override applies only to chat generation (streaming and text completions); JSON-object completions (reranker, enrichment) always inherit the gateway/model default so reasoning tokens cannot starve their small output budgets. Java Chat forwards the universal reasoning values `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max` without model inspection; the gateway owns upstream capability translation. A deterministic gateway `HTTP 422 unpreservable_reasoning_intent` means no provider can preserve the configured effort — lower or unset `reasoning-effort`; retrying unchanged cannot succeed.
Non-secret generation policy is owned by `app.llm` in `application.properties`: `temperature`, `reasoning-effort`, `completion-output-token-budget`, `enrichment-output-token-budget`, `reranker-temperature`, `reranker-output-token-budget`, and `configured-provider-backoff-seconds`. The tested default is Claude Haiku 5.5 with `reasoning-effort=low`, which preserved quality while reducing latency and cost in the Java Chat matrix; both temperature settings stay at `1.0` and the gateway owns provider-specific sampling constraints. The override applies only to chat generation (streaming and text completions); JSON-object completions (reranker, enrichment) pin reasoning to `none` so reasoning tokens cannot starve their small output budgets. Java Chat forwards the universal reasoning values `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max` without model inspection; the gateway owns upstream capability translation. A deterministic gateway `HTTP 422 unpreservable_reasoning_intent` means no provider can preserve the configured effort — lower or unset `reasoning-effort`; retrying unchanged cannot succeed.

### Shared LLM gateway

Expand Down
Loading