Repository navigation
Promote measured Haiku model and gateway workload attribution #259
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
3a7ad4d
049db3d
cde5608
9372902
a0009b7
929bdf6
06c53a0
fe7f545
19ef5e8
b5b94e4
d47969c
9f37237
b1d4e5a
e399bc6
2a740a3
396452d
7989641
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,10 +19,10 @@ Common variables: | |
|
|
||
| - `OPENAI_API_KEY` (shared-gateway auth) | ||
| - `OPENAI_BASE_URL` (`https://api.llm-gateway.iocloudhost.net/v1` for Java Chat deployments) | ||
| - `OPENAI_MODEL` (non-blank gateway chat alias; default `gpt-5.4`) | ||
| - `OPENAI_MODEL` (non-blank gateway chat alias; default `claude-haiku-5-5`) | ||
| - `OPENAI_STREAMING_REQUEST_TIMEOUT_SECONDS` (default `90`; bounds the complete SDK call and the visible-output deadline, which shares this same budget) | ||
|
|
||
| Non-secret generation policy is owned by `app.llm` in `application.properties`: `temperature`, optional `reasoning-effort`, `completion-output-token-budget`, `enrichment-output-token-budget`, `reranker-temperature`, `reranker-output-token-budget`, and `configured-provider-backoff-seconds`. `gpt-5.4` is a reasoning generation whose upstream accepts only the default temperature (`1.0`) on the Responses API, so both temperature settings stay at `1.0`; any other value fails upstream with `HTTP 400`. `reasoning-effort` is absent by default, preserving the gateway/model default; set it only for an explicit override. Invalid values fail startup. The override applies only to chat generation (streaming and text completions); JSON-object completions (reranker, enrichment) always inherit the gateway/model default so reasoning tokens cannot starve their small output budgets. Java Chat forwards the universal reasoning values `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max` without model inspection; the gateway owns upstream capability translation. A deterministic gateway `HTTP 422 unpreservable_reasoning_intent` means no provider can preserve the configured effort — lower or unset `reasoning-effort`; retrying unchanged cannot succeed. | ||
| Non-secret generation policy is owned by `app.llm` in `application.properties`: `temperature`, `reasoning-effort`, `completion-output-token-budget`, `enrichment-output-token-budget`, `reranker-temperature`, `reranker-output-token-budget`, and `configured-provider-backoff-seconds`. The tested default is Claude Haiku 5.5 with `reasoning-effort=low`, which preserved quality while reducing latency and cost in the Java Chat matrix; both temperature settings stay at `1.0` and the gateway owns provider-specific sampling constraints. The override applies only to chat generation (streaming and text completions); JSON-object completions (reranker, enrichment) always inherit the gateway/model default so reasoning tokens cannot starve their small output budgets. Java Chat forwards the universal reasoning values `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max` without model inspection; the gateway owns upstream capability translation. A deterministic gateway `HTTP 422 unpreservable_reasoning_intent` means no provider can preserve the configured effort — lower or unset `reasoning-effort`; retrying unchanged cannot succeed. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Document the explicit JSON reasoning setting. JSON-object completions do not inherit the gateway/model reasoning default. 🤖 Prompt for AI AgentsSource: Coding guidelines |
||
|
|
||
| ### Shared LLM gateway | ||
|
|
||
|
|
@@ -32,7 +32,7 @@ Configure chat through the shared gateway with: | |
| OPENAI_API_KEY=lgw-... | ||
| OPENAI_BASE_URL=https://api.llm-gateway.iocloudhost.net/v1 | ||
| # Default; any non-blank gateway alias is supported. | ||
| OPENAI_MODEL=gpt-5.4 | ||
| OPENAI_MODEL=claude-haiku-5-5 | ||
| ``` | ||
|
|
||
| Java Chat sends chat and embedding requests to the same configured gateway URL and credential. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,22 @@ | ||
| # Every lane runs on the Mac build host. Any other host (the laptop, Linux peers) forwards the goals | ||
| # and command-line variables, each shell-quoted, to a warm mirror of this checkout there through the | ||
| # host tool remote-slot (dotfiles, ~/.local/bin). The environment, gitignored files, and make's own | ||
| # flags do not cross, and every output (installs, archives, receipts) stays on that host. GitHub | ||
| # Actions jobs already run on the Mac their workflow picked. | ||
| APPLE_BUILD_HOST := haiku-7 | ||
| ifeq ($(filter $(APPLE_BUILD_HOST),$(shell hostname -s))$(GITHUB_ACTIONS),) | ||
| APPLE_BUILD_HOST_VARIABLES := $(strip $(foreach variable,$(sort $(.VARIABLES)),$(if $(filter command line,$(origin $(variable))),'$(variable)=$(subst ','\'',$(value $(variable)))'))) | ||
| .DEFAULT_GOAL := apple-build-host | ||
| .PHONY: apple-build-host $(MAKECMDGOALS) | ||
| apple-build-host: | ||
| remote-slot run $(APPLE_BUILD_HOST) -- make $(MAKECMDGOALS) $(APPLE_BUILD_HOST_VARIABLES) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When Useful? React with 👍 / 👎. |
||
| ifneq ($(MAKECMDGOALS),) | ||
| $(MAKECMDGOALS): apple-build-host ; @: | ||
| endif | ||
| else | ||
|
|
||
| .PHONY: \ | ||
| signing-session \ | ||
| desktop \ | ||
| desktop-test \ | ||
| desktop-archive \ | ||
|
|
@@ -12,6 +30,9 @@ | |
| ios-archive \ | ||
| ios-export-app-store | ||
|
|
||
| # Every xcodebuild lane compiles through Xcode's shared per-user compilation cache. | ||
| export XCODE_XCCONFIG_FILE := $(CURDIR)/compilation-cache.xcconfig | ||
|
|
||
| DEVELOPMENT_TEAM := NB356A75PU | ||
| ASC_PROFILE := csweb | ||
| DEVELOPER_ID_APPLICATION := Developer ID Application: William Callahan (NB356A75PU) | ||
|
|
@@ -27,6 +48,12 @@ IOS_DISTRIBUTION_DIR := iosApp/distribution | |
| IOS_ARCHIVE := $(IOS_DISTRIBUTION_DIR)/JavaChat.xcarchive | ||
| IOS_APP_STORE_EXPORT := $(IOS_DISTRIBUTION_DIR)/app-store | ||
|
|
||
| # Signed lanes unlock this Mac's build keychain through the host tool build-keychain (dotfiles, | ||
| # ~/.local/bin), so they run from a fresh SSH session with no console login [HH0a]; a Mac without | ||
| # one signs from its open login keychain. CI never runs a signed lane. | ||
| signing-session: | ||
| @build-keychain unlock | ||
|
|
||
| desktop: | ||
| xcodebuild \ | ||
| -project desktopApp/JavaChatDesktop.xcodeproj \ | ||
|
|
@@ -45,7 +72,7 @@ desktop-test: | |
| -derivedDataPath desktopApp/build-test \ | ||
| test CODE_SIGNING_ALLOWED=NO | ||
|
|
||
| desktop-archive: | ||
| desktop-archive: signing-session | ||
| xcodebuild \ | ||
| -project desktopApp/JavaChatDesktop.xcodeproj \ | ||
| -scheme JavaChatDesktop \ | ||
|
|
@@ -72,7 +99,7 @@ desktop-export-developer-id: desktop-archive | |
| desktop-dmg: desktop-export-developer-id | ||
| $(MAKE) desktop-create-signed-dmg | ||
|
|
||
| desktop-create-signed-dmg: | ||
| desktop-create-signed-dmg: signing-session | ||
| rm -rf "$(DESKTOP_DMG_STAGING)" | ||
| mkdir -p "$(DESKTOP_DMG_STAGING)" | ||
| ditto \ | ||
|
|
@@ -135,7 +162,7 @@ ios-test: | |
| -derivedDataPath iosApp/build-test \ | ||
| test CODE_SIGNING_ALLOWED=NO | ||
|
|
||
| ios-archive: | ||
| ios-archive: signing-session | ||
| xcodebuild \ | ||
| -project iosApp/JavaChat.xcodeproj \ | ||
| -scheme JavaChat \ | ||
|
|
@@ -152,3 +179,4 @@ ios-export-app-store: ios-archive | |
| -exportPath $(IOS_APP_STORE_EXPORT) \ | ||
| -exportOptionsPlist iosApp/ExportOptions-AppStore.plist \ | ||
| -allowProvisioningUpdates | ||
| endif | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| // Loaded by every Make lane through XCODE_XCCONFIG_FILE, so package targets are | ||
| // cached too. The cache sits at Xcode's per-user default, not in each lane's | ||
| // DerivedData, so a lane rebuilt from an empty DerivedData replays what it | ||
| // compiled before, and every lane and checkout reuses compiled packages. | ||
| COMPILATION_CACHE_ENABLE_CACHING = YES | ||
| COMPILATION_CACHE_CAS_PATH = $(HOME)/Library/Developer/Xcode/DerivedData/CompilationCache.noindex |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sentence says reranker and enrichment requests inherit the gateway/model reasoning default, but the same commit changes
OpenAiRequestFactory.configureResponseParamsto sendReasoningEffort.NONEwheneverrequireJsonObjectis true. With the new application default oflow, operators troubleshooting request behavior are therefore told that these requests uselowwhen their wire payload explicitly usesnone; update this description to match the emitted request.Useful? React with 👍 / 👎.