Skip to content

Pick clips in a pass of their own, one video at a time - #25

Merged
davior merged 1 commit into
mainfrom
claude/trusting-ramanujan-54etzj
Oct 28, 2026
Merged

davior merged 1 commit into
mainfrom
claude/trusting-ramanujan-54etzj

Conversation

@davior

@davior davior commented Oct 28, 2026

Copy link
Copy Markdown
Owner

Why

Since #24 the assistant can read whole transcripts, but it reads them inside its own conversation. Everything it reads there is sent again on every later round, so weighing several long interviews gets expensive quickly. The model also has to judge passages while it is busy writing a video.

What changes

  • find_clips(gam_id, idea, count=3) is a new assistant tool. It hands one video's whole transcript (numbered sentences) and the idea a scene needs to a separate model call with no tools, using the user's own provider. Only the best passages come back to the request, each with:
    • a ready trim for set_visual,
    • what is said,
    • a score from 1 to 10,
    • one sentence on why it fits.
  • The picker names line numbers, never times. Times come from the transcript, so a cut can't land on a moment the model made up. Passages naming lines the picker wasn't shown, or longer than 90 s, or scored below 4, are dropped.
  • Long videos are read in overlapping windows of about 15k tokens. The best passages across all windows are kept, with overlapping ones removed.
  • Cost controls:
    • Picker calls are capped per request with ASSISTANT_MAX_CLIP_PICKS (default 16). Over the cap, the tool tells the model to use read_transcript instead.
    • Results are cached per video and idea within a request.
    • Picker calls are billed to the request like any other model call.
  • Prompt workflow: search for candidates, find_clips each, compare across videos, then set_visual with the chosen trim. read_transcript stays for checking context, or as a fallback.
  • Shared padding: the padding logic (_padded) is shared by the automatic hit trim and the picked clips.

No GAM changes are needed.

Testing

  • pytest -q: 719 passed. New tests cover:
    • ready trims and what the picker is shown,
    • caching,
    • rejecting lines the picker didn't read,
    • windowed reading of long videos,
    • the call cap,
    • missing picker or transcript, and a bad count,
    • de-duplicating overlapping passages,
    • a whole request through the worker, checking the pass runs as its own tool-less conversation and is billed to the request.
  • npm test && npm run build: 187 passed and the build succeeds (no frontend changes).

Generated by Claude Code

find_clips hands one video's whole transcript, numbered by sentence, and
the idea a scene needs to a separate model call with no tools, and gives
the turn back only the best passages: a ready trim, what is said, a score
and why. The turn's own conversation no longer has to carry every
transcript it weighs on every round, and the picker judges passages with
nothing else to do.

The picker names lines, never times, so a cut can't land on a moment it
made up; a passage naming lines it wasn't shown is dropped. Long videos
are read in overlapping windows of about 15k tokens and the best of all
kept. Calls are capped per request (ASSISTANT_MAX_CLIP_PICKS, 16), cached
per video and idea, and billed to the turn like any other call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1geD2rbaPtRa9JcN8sZH8
@davior
davior marked this pull request as ready for review October 28, 2026 10:55
@davior
davior merged commit 07b88e4 into main Oct 28, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants