Skip to content

[WIP] Introduce capability-aware native tool-calling runtime - #4315

Draft
JasonW404 wants to merge 22 commits into
developfrom
feature/native-toolcalling-runtime
Draft

JasonW404 wants to merge 22 commits into
developfrom
feature/native-toolcalling-runtime

Conversation

@JasonW404

@JasonW404 JasonW404 commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Native chat can now use the develop model service for real reasoning and native tools while retaining shared context maintenance. Normal content-only responses finish only on stop; text accompanying tool calls remains intermediate. Unsupported or unknown tool capability falls back to CodeAgent per selected model, and missing reasoning capability does not request simulated ReAct reasoning.

Depends on #4321 (shared non-blocking context maintenance), merged into this branch after its local built-container acceptance. Review/merge the shared PR first. Both PRs target develop; the separate v2.8.4 branch remains standalone.

The integration records provider input usage against the actual request/profile, preserves native call/result order during result compaction, binds owned original outputs to persisted references, exposes direct/programmatic readers, and performs bounded rebuild only for explicit provider input overflow without replaying completed effects. Estimates and failed maintenance do not independently reject chat or an Agent run. Output length remains an incomplete generation, not generic input overflow.

Validation:

  • Shared PR CI green: 23,797 passed / 32 skipped / zero failed; Codecov patch 92.24806%.
  • Latest native/legacy focused unit selection: 127 passed; additional context/history/execution regression selection: 142 passed; additional shared/context and isolated factory/observation regressions passed separately.
  • Maintained controlled HTTP/SSE D3: 13 passed, cleanup 0; independently rerun against the installed SDK with source identity checks.
  • Fresh real-model SDK D3: 17 passed; all 98 spans across 17 traces inspected in Phoenix default.
  • Final deployed API: native/fallback/denial D2 (5), actual GLM/Qwen reasoning D2 (2), image/source/health D5 (1).
  • Actual native/MCP/Python/programmatic/file tools; KB and chunk CRUD plus Agent search; short-term memory CRUD/search/prefetch/store; Workbench generic/single/multi and Skill mounts; NL2Agent draft saving and NL2Skill generation/install/read.
  • Persisted long run: 30 tasks / 31 user turns / 98 steps / 62 sequential completed messages, five accepted history checkpoints, two original-output offload/load cycles with one producer effect each; all 29 exact ledger pairs plus KB/memory facts recovered. One provider promise-only stop required an explicit follow-up; completed effects were not replayed. A direct-only harness assertion was corrected after verifying actual programmatic MCP execution.
  • First integration CI found one legacy export compaction regression among 23,964 tests. Corrected the format distinction in 5dd1ae2 and reproduced/passed the original test. Rebuilt and repeated installed controlled scenarios (13), real native/fallback D2 (5), D5 identity/health, real pressure-triggered offload/read and browser retained history. The 30-task campaign was run on product 9102cde; the compatibility fix preserves native projection behavior and is covered by these relevant reruns. Built final main image from completed branch; all 30 changed backend/SDK product files match installed hashes. No product source mounts. Host deployment remains native with Phoenix default. Browser reload/reopen confirms saved history and final ledger.

Limits/findings: conservative fact guards can reject summaries and add latency while preserving original history. Supplementary reruns also recorded a fallback model returning unexecuted Python in its final answer (one bounded fresh D2 rerun passed) and a too-small offload fixture: no reference was issued below threshold, the model guessed references and repeated producers until max steps. That failed chat/trace is retained; a fresh pressure-triggered fixture passed with one producer and one real read. These failures are not counted as passes. The long run encountered an external embedding timeout and an auxiliary memory-extraction LLM failure; chat still completed. Full long prompts exceed the existing 100k-character Phoenix input-preview bound; complete final output, persisted history, context fingerprints and budget provenance were verified separately. This is representative feature coverage, not exhaustive certification of every UI, multimodal/provider, long-term memory or dreaming path. Existing authoring services intentionally retain CodeAgent through the new model service. Full-Case semantic-review status remains distinct from mapped formal execution.

Test chats, Agent/model/KB/MCP/Skill/memory resources and Phoenix traces are retained for review at the user's request. Agent 6 v2; long-run conversation 14; model 32; embedding 33; fallback model 34; MCP 1; Skill 7. Current evidence and limitations are maintained in the workspace SPEC execution record.

Final CI on 5dd1ae2: all 16 checks passed; 23,933 unit tests passed, 32 skipped, zero failed; Codecov patch 93.01471%. Both image architectures, type checking, copyright and CodeQL are green. Final host main image 1d8637b4...; 16 native test conversations and all persistent fixture resources/traces retained.

Reasoning visibility recheck: the retained long-run campaign used test outlet 32 with enable_thinking=false. Enabled only that owned setting (saved revision 3), preserving output/context budgets and Agent definitions. Ordinary chats 20–23 now retain real Qwen/GLM reasoning, including MCP/Python calls; stream and saved reasoning hashes match, and both providers render Reasoning cards after reload/reopen. Source/image remain 5dd1ae2/1d8637b4..., with no additional product code change. All original chats/resources and default Phoenix traces remain retained.

Native stream compatibility follow-up: ordinary model text streams as model_output_thinking, provider reasoning streams as model_output_deep_thinking, and a successful text-only stop emits the full accepted body in final_answer. Removed the experimental model_output chat path. A one-time PostgreSQL update on the test host converted 18 saved message units across seven chats, verified their other fields and left zero rows with the experimental type. History API, browser, installed HTTP/SSE scenarios and Qwen on Phoenix default confirm the deployed behavior. All 16 checks pass on f68a742 with 23,933 tests passed, 32 skipped and 93.00613% patch coverage.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant