Skip to content

Feature: stream replies into the transcript and give text before a tool call its own card - #78

Merged
DevMando merged 3 commits into
mainfrom
feature/stream-reply-cards
Sep 25, 2026
Merged

DevMando merged 3 commits into
mainfrom
feature/stream-reply-cards

Conversation

@DevMando

Copy link
Copy Markdown
Owner

Summary

Agent replies now appear in the transcript while they're being written, instead of all at once when the model finishes. When the agent says something before using a tool ("let me check the docs…"), that text gets its own card above the tool call, and the answer gets a second card below it. Before, both parts were merged into one card at the end.

What changed

  • Live draft: while a reply streams, its text shows in a draft card at the bottom of the transcript. It updates about ten times a second and turns into the normal formatted card when the reply finishes. The draft is never saved to the session history; only the finished cards are.
  • Split at tool calls: when the model starts a tool call mid-reply, the text so far becomes its own card, ahead of the tool pill.
  • Protection against wrong text: some local models write tool calls as plain text, which the engine removes only after the reply ends. Cards made early are therefore provisional. They're kept only if the final reply contains their text, and removed otherwise, so raw tool-call text never stays on screen.
  • Background tabs: an agent working in a tab that hasn't been opened yet queues its cards in the right order.
  • Engine pin: the submodule moves to engine main at 9904aad, which contains the streaming hooks this needs (Feature: print CLI replies in the order they happened, and fix loose numbered lists MandoCode#99), plus #100 (tok/s and cut-off notices, which Desktop's reply summary also benefits from) and #101 (the CLI UI work).

Why this was done

Long replies used to leave the transcript empty until the model finished, which read as if nothing was happening. And a sentence the agent wrote before looking something up was glued to the answer it wrote afterwards, so the conversation didn't read in the order it happened.

How it was tested

  • New unit tests cover the split at tool calls, the case where the final text was rewritten (provisional cards removed), draft updates, and the no-streaming case. The full Desktop test suite passes.
  • Worth a manual check in the app before merging: ask a question that triggers a web search and confirm the text streams in, the lead-in gets its own card above the tool pill, and the answer lands below it.

Notes

  • This only applies when response streaming is on for the model, which is the default. With streaming off, a reply still arrives as one card.
  • The draft is plain text and updates on a timer, never per token; it's cheap to redraw and stays static, with no animation. The E-Ink theme's grayscale filter is turned off on the draft for the same reason.
  • When a session is restored, a card made before a tool call appears after that call's tool pills rather than before them. The live view is correct; only the restored order differs.
  • The engine's /history command now appears in Desktop's slash-command suggestions but isn't handled here (it reports "Unknown command"). Desktop's transcript already has Ctrl+F search.
  • Changelog: entry added under [Unreleased] → Added.

DevMando and others added 3 commits September 24, 2026 14:02
While a turn streams, its text shows in a live draft card that repaints
about ten times a second and is never journaled. When the model starts a
tool call mid-reply, the text so far becomes its own card ahead of the
tool pill, so words written before a tool call no longer merge into the
answer written after it. Those cards stay provisional until the turn
settles: they are kept only if the final text contains them, in order,
and removed otherwise, so a text-written tool call never stays on screen.

Depends on the engine's OnResponseTextDelta, IsStreamingModelCall and
ReplyText (DevMando/MandoCode feature/stream-reply-split); the submodule
points at that branch until it merges.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Points the MandoCode submodule at engine main 9904aad, which carries
DevMando/MandoCode#99 (reply streaming hooks and ReplyText), #100 (tok/s
and cut-off notices while streaming) and #101 (CLI live UI and
ContextMeter).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@DevMando
DevMando merged commit 751e978 into main Sep 25, 2026
1 check passed
@DevMando
DevMando deleted the feature/stream-reply-cards branch September 25, 2026 00:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant