Skip to content

Let the assistant read whole transcripts before choosing a clip - #24

Merged
davior merged 1 commit into
mainfrom
claude/trusting-ramanujan-54etzj
Oct 28, 2026
Merged

davior merged 1 commit into
mainfrom
claude/trusting-ramanujan-54etzj

Conversation

@davior

@davior davior commented Oct 27, 2026

Copy link
Copy Markdown
Owner

Why

When the assistant picked clips, all it saw was the search result: one moment per video, the best match for its query words (a single transcriber segment plus a 240-character snippet). Transcriber segments end wherever the speaker pauses, often mid-sentence, so the cuts started and stopped mid-thought. Any trim the assistant wrote itself was a guess, because it had never seen the rest of the transcript.

What changes

  • read_transcript(gam_id, start?, end?) is a new assistant tool that calls GAM's existing GET /api/assets/{id}/transcript.
    • It regroups the words into sentences, one line each: [start-end] text.
    • It labels speakers when there is more than one, and treats a segment corrected by hand in GAM as its text.
    • Long transcripts come back in pages of about 30k characters, each ending with the next start to read from.
    • Each transcript is fetched at most once per turn.
  • Clip-choosing workflow in the prompt. Search to find candidate videos, read the most promising transcripts (several at once), choose the passage that best demonstrates the idea, then set_visual with a trim taken from the transcript lines.
  • The automatic trim from a hit grows to the whole sentences it falls in. Growth is capped at 12 s each way. The usual 0.5 s padding now stops where the neighbouring sentence begins. If no transcript is available, the trim behaves as before.
  • more_moments on search hits passes through GAM's other_matches, so the assistant knows the video matched in other places too.

No GAM changes are needed.

Testing

  • pytest -q: 712 passed. New tests cover sentence regrouping, speaker labels, paging and caching, untranscribed and unknown assets, hand-edited segments, sentence-snapped trims and more_moments.
  • npm test && npm run build: 187 passed and the build succeeds (no frontend changes).

Generated by Claude Code

A search shows one moment per video, a single transcriber segment that
often starts or ends mid-sentence, and the assistant had nothing else to
cut from. It now has read_transcript, which reads a GAM transcript as
whole sentences with their times (paged for long videos), and the prompt
tells it to search for candidates, read them, then cut the passage that
best makes the point.

A hit's automatic trim grows to the whole sentences it falls in, padded
without reaching into the next one, and a hit says when more of the
video matched (GAM's other_matches).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1geD2rbaPtRa9JcN8sZH8
@davior
davior marked this pull request as ready for review October 28, 2026 06:36
@davior
davior merged commit 5ec48ba into main Oct 28, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants