Skip to content

feat: metrics page backed by a durable usage rollup - #165

Merged
sanchitrk merged 8 commits into
canaryfrom
claude/desktop-metrics-page-b1eb96
Nov 14, 2026
Merged

sanchitrk merged 8 commits into
canaryfrom
claude/desktop-metrics-page-b1eb96

Conversation

@sanchitrk

@sanchitrk sanchitrk commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

What

A Metrics page in the desktop app (chart button beside Settings in the sidebar), backed by usage the harness now records durably and the app server mirrors per machine.

How it fits together

  1. Harness keeps two tables, written in the same transaction that finalises an entry:
    • usage: per UTC hour × thread × provider × model × thinking level — requests, tokens by type, reasoning tokens, cost by type, summed response time and a fixed-edge duration histogram.
    • tally: per hour × thread — user messages and tool calls that ran, by name.
      Rows copy the session and project ids and have no foreign key, so they outlive a deleted session. A migration backfills existing entries with the same code as the live path.
  2. CodeWork RPC serves the rows as usage.sync (revision order, then live; running totals a client replaces by key) and reports the machine's own name in server.info.
  3. App server follows usage.sync per machine into its own tables, keyed by machine and store, so history also survives a recreated harness database. It serves metrics.usage and metrics.hours by calendar day and hour in a given time zone, with machine and project filters.
  4. Web renders the page from one year of usage and refreshes as usage syncs.

Also in this PR

  • Machine names come from the machine. The name typed in Settings is now an optional override; the built-in server is no longer seeded as "CodeWork".

Correctness notes

  • An assistant entry is rolled up exactly once, when it becomes final. Per thread, the rollup's tokens and cost equal the totals already kept on the thread row; tests assert it.
  • Aborted, failed and crash-recovered requests are billed and counted, but never timed, so a recovery after downtime cannot inflate response time.
  • Tool calls count only when they ran (completed or errored).
  • Response time on the page is an estimate from the duration histogram (median and p95 accurate to one bucket); the page says so.
  • usage.sync is a stream of its own rather than new index.sync variants, so a desktop that predates it keeps syncing; an app server talking to an older CodeWork server simply shows no usage for that machine.

Testing

  • vp run typecheck and vp check: clean.
  • Full suites with Anthropic, OpenAI and GitHub Copilot credentials: packages/harness 658 passed / 34 skipped; packages/codework 159 passed / 19 skipped; apps/server all non-live tests passed.
  • Live OrbStack machine suites, run one at a time: usage.machines.live 6/6 (new), sync.machines.live 12/12, daemon.machines.live 1 passed / 1 skipped, recovery.machines.live 5/5.
  • Checked in the Electron app with real prompts: the page's totals match the harness thread totals.

Known limits

  • Run together, the live machine suites collide on the shared machines; they pass serially.
  • reads.test.ts "each read's topics fire…" times out intermittently, with or without this change.
  • Not looked at on screen: the Machine filter with more than one machine, the folded "Deleted Sessions" row, narrow widths.
  • Session rows have no sandbox label yet (the app does not know the driver), and nothing removes old usage rows yet; both are left for later Settings work.
  • After an app-server restart the page's error state needs a manual Retry.

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

sanchitrk and others added 8 commits November 14, 2026 14:16
Two tables written in the transaction that finalises an entry: usage (requests, tokens by type, reasoning, cost by type, response time and a duration histogram per UTC hour, thread, provider, model and thinking level) and tally (user messages and tool calls that ran). Rows carry the session and project they were written for and no foreign key, so they outlive a deleted session. Aborted requests are billed and counted but never timed. A migration backfills from existing entries with the same code as the live path.

Sync.usage reads both tables by revision for a follower.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
usage.sync streams the usage rollup in revision order, then live, as running totals a client replaces by key. It is a stream of its own so a client that predates it is unaffected.

server.info gains an optional host name: the macOS computer name, else the hostname.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…serve metrics

The app server follows usage.sync for each machine into its own usage and tally tables, keyed by machine and store so history survives a recreated store, and keeps working against a server without the stream.

A machine's name is now the one its server reports; the name typed in Settings is an optional override, and the built-in server is no longer seeded as CodeWork.

metrics.usage and metrics.hours read the mirror by calendar day and hour in a given time zone, with machine and project filters.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A Metrics page beside Settings in the sidebar: cost, tokens and sessions for a period; a 12-month activity calendar by prompts, sessions, tokens or cost with a per-day view; tokens or cost over time by model or provider; projects, models, response time, weeks, tool calls, thinking levels, cache and top sessions. It reads one year of usage and refreshes as usage syncs.

The server name in Settings is optional and shows the machine's own name when empty.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ross navigation

Tooltips live in their own store, so following the pointer re-renders only the
tooltip; the calendar is sized before its first paint and keeps its scroll on a
refresh. The year's read outlives the page for a minute and the period and
filters persist across navigation, a removed machine's filter falling back to
every machine. Change badges carry direction in the arrow, the cache badge
shows points, and a model id shared by two providers is labelled provider first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…server's usage

Each open of the harness database records a boot with the revision it opened
at. usage.sync takes the client's last boot and, when the store left that line
(a restored backup), reads from zero with reset; a read from zero replaces the
store's mirrored rows in one transaction. The app server keys usage by store
with no foreign key to machine, so a removed server's history stays and the
server added again replaces it rather than adding to it. Metrics also refresh
on index changes, for session titles and deletions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…achines

seed.ts writes a history of sessions, models and tools through the thread store;
the live suite checks the metrics reads against each machine's own totals across
live turns, app and machine restarts, a paused machine, a network cut, a
restored backup, a server removed and added again, and a recreated store. The
fixture model can stamp replies with the real time (CODEWORK_FIXTURE_NOW=1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The machine suites run it inside each OrbStack machine by path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sanchitrk
sanchitrk merged commit 6482470 into canary Nov 14, 2026
1 check passed
@sanchitrk
sanchitrk deleted the claude/desktop-metrics-page-b1eb96 branch November 14, 2026 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant