diff --git a/README.md b/README.md index cffbaf7..985de4d 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ Most editors keep a project in a format built for their own app, not for you or - **Timeline editor**: a main track of clips that play end to end, text on overlay tracks (colour, box and outline, for titles and captions alike), music on audio tracks. Trim, split, reorder and zoom, with filmstrips and waveforms. - **Live preview**: one master clock plays the cut back in the browser as you edit. - **Footage from a Drive folder**: start a project from a Google Drive folder shared with the link, and every video in it and in the folders inside it comes in as that project's own footage. Each clip is logged as it lands: what it shows, the best lines said with their times, the best moments to cut away to, and anything readable on screen. It runs in the background, so an agent can read a whole shoot's logs and choose what goes in. -- **Highlights**: every logged clip is read against what the video is for. You get the soundbites and the b-roll worth an editor's look, each scored with the reason why, and the clips not worth anyone's time set aside with theirs. Review them on one page where each pick plays from its in to its out, keep or drop with a key, and export what you keep as a timeline for Premiere Pro or DaVinci Resolve (the picks end to end on the camera files, nothing re-encoded) or as a sheet. Ask for a change uses them too: "make a 60 second cut from the kept soundbites". +- **Highlights**: every logged clip is read against what the video is for. You get the soundbites and the b-roll worth an editor's look, each scored with the reason why, and the clips not worth anyone's time set aside with theirs. Say how many you want, and once every clip is read the best that many are chosen from all of them, as a set: the mix the brief asks for, never the same moment twice, and when you drop one the next comes in. Review them on one page where each pick plays from its in to its out, keep or drop with a key, and export what you keep as a timeline for Premiere Pro or DaVinci Resolve (the picks end to end on the camera files, nothing re-encoded) or as a sheet. Ask for a change uses them too: "make a 60 second cut from the kept soundbites". - **Media library**: upload clips, stills and music, or import them from Google Drive, and use them in any project. Videos go straight to the managed media service, resumably and up to 30 GB, so a multi-gigabyte master plays, gets a transcript and exports without passing through the app. Each upload shows its progress, and one that drops picks up where it stopped. - **Formats**: 16:9, 9:16 for Reels, TikTok and Shorts, 1:1, 4:5 and 4:3, or the shape of your own footage. Titles and logos keep their size and place when the shape changes, and clips either fit with bars or fill the frame. - **Crop**: keep part of any clip, logo or picture-in-picture, free or held to 16:9, 9:16, 1:1, 4:5 or the video's own shape, with a scrubber to check the subject stays inside. The preview places a cropped clip exactly as the export does. diff --git a/agent.md b/agent.md index 082daa1..74f21b8 100644 --- a/agent.md +++ b/agent.md @@ -114,16 +114,31 @@ and the reason. They are read from the logs and transcripts in the background, a folder at a time, and clips logged later join by themselves. ``` -POST /api/projects/{id}/highlights { "brief"?: "…", "again"?: true } +POST /api/projects/{id}/highlights { "brief"?: "…", "again"?: true, "wanted"?: 25 } +PUT /api/projects/{id}/highlights/wanted { "wanted": 25 | null } how many; nothing is read again POST /api/projects/{id}/highlights/stop stop reading; picks so far stay, asking again resumes GET /api/projects/{id}/highlights best first, 100 a page (limit up to 1000, offset) &kind=soundbite|broll &min_score=2..5 &folder=Day 1/Cam B - &pick=open|keep|drop|not_dropped|all &sort=score|clip &skipped=1 + &pick=best|open|keep|drop|not_dropped|all &sort=score|clip &skipped=1 PATCH /api/projects/{id}/highlights/{hid} { "pick"?: "keep"|"drop"|null, "start"?, "end"?, "speaker"?, "text"? } POST /api/projects/{id}/highlights/items { "clip": "", "start"?, "end"?, "kind"?, "text"? } -GET /api/projects/{id}/highlights/export?format=csv|xml&pick=keep|not_dropped&root=&fps= +GET /api/projects/{id}/highlights/export?format=csv|xml&pick=best|keep|not_dropped&root=&fps= ``` +`wanted` is how many highlights the video wants (1 to 200; null, the default: +every pick worth a look). Write the number there, never in the brief: each +reading sees a dozen clips, so a count in the brief only holds for each dozen. +With a number, every clip is still read, then one more model call goes over +all the picks and puts them in order for the brief: the best set of that many +first (the mix the brief asks for, one clip may give most of it, never the +same moment twice), then stand-ins. `pick=best` lists that set: kept picks +first (they count toward the number; when more are kept than wanted, the best +of them), then the model's order. Dropping one brings in the next with no new +call. A new number orders them again; picks from clips read later do too, at +most every 15 minutes while footage is still coming in. The list says where it +stands: `wanted`, `counts.best`, and `ranking: { status: waiting | running | +done | failed, error, due }`. + `again` reads every clip again; what a person kept or dropped stays as it is, and each reading learns from those calls (the latest 15 kept and 15 dropped) beside the brief. A `PATCH` with `pick` is how you teach it. @@ -213,7 +228,8 @@ in line, and asks Drive again now for the clips waiting on it. `DELETE /api/proj | POST | `/api/projects/{id}/footage/folders` | `{ url, language? }` adds a Drive folder shared with the link | | POST | `/api/projects/{id}/footage/sync` | Takes in files added to the folders since → `{ added }` | | POST | `/api/projects/{id}/footage/retry` | Puts failed imports and logs back in line → `{ imports, logs }` | -| POST | `/api/projects/{id}/highlights` | Find the footage's highlights `{ brief?, again? }` (in the background) | +| POST | `/api/projects/{id}/highlights` | Find the footage's highlights `{ brief?, again?, wanted? }` (in the background) | +| PUT | `/api/projects/{id}/highlights/wanted` | How many highlights `{ wanted: 1..200 \| null }`: the best that many are chosen from all the picks | | POST | `/api/projects/{id}/highlights/stop` | Stop reading: clips in line go back, later clips wait until asked again | | GET | `/api/projects/{id}/highlights` | The highlights, best first, filtered and paged → `{ clips, counts, highlights, next_offset, paused }` | | PATCH | `/api/projects/{id}/highlights/{hid}` | A person's call `{ pick?: keep\|drop\|null, start?, end?, speaker?, text? }` | diff --git a/schema.sql b/schema.sql index 8b51b8b..f2611b2 100644 --- a/schema.sql +++ b/schema.sql @@ -230,3 +230,16 @@ ALTER TABLE project_footage ADD COLUMN upload_url TEXT; ALTER TABLE project_footage ADD COLUMN upload_expires TEXT; ALTER TABLE project_footage ADD COLUMN upload_size INTEGER; ALTER TABLE project_footage ADD COLUMN upload_done INTEGER; + +-- How many highlights the project wants; NULL: every pick worth a look. Once +-- every clip is read, one more model call puts the picks in order for the +-- brief, the best set of that many first (src/server/highlights.ts). The +-- ordering: NULL | waiting | running | done | failed, with the error, and +-- when it was last claimed (a claim older than five minutes is taken over). +ALTER TABLE edit_projects ADD COLUMN highlights_wanted INTEGER; +ALTER TABLE edit_projects ADD COLUMN highlights_rank_status TEXT; +ALTER TABLE edit_projects ADD COLUMN highlights_rank_error TEXT; +ALTER TABLE edit_projects ADD COLUMN highlights_rank_at TEXT; +-- A pick's place in that order, 1 the best. The best are the kept picks, +-- then the first in this order; NULL: not placed yet. +ALTER TABLE footage_highlights ADD COLUMN rank INTEGER; diff --git a/src/client/highlights.tsx b/src/client/highlights.tsx index 162a56d..4286ff9 100644 --- a/src/client/highlights.tsx +++ b/src/client/highlights.tsx @@ -11,6 +11,7 @@ import { Check, ChevronRight, Download, + ListOrdered, Loader2, RotateCcw, Sparkles, @@ -61,7 +62,11 @@ interface HighlightList { stopped: boolean; brief: string; clips: { logged: number; done: number; pending: number; failed: number; skipped: number; not_asked: number }; - counts: { total: number; soundbites: number; broll: number; kept: number; dropped: number; open: number }; + counts: { total: number; soundbites: number; broll: number; kept: number; dropped: number; open: number; best: number }; + /** How many highlights the video wants; null: every pick worth a look. */ + wanted: number | null; + /** Choosing the best `wanted`: `due`, it runs on the next step. */ + ranking: { status: "waiting" | "running" | "done" | "failed" | null; error: string | null; due: boolean } | null; paused: string | null; highlights: Highlight[]; next_offset: number | null; @@ -71,12 +76,24 @@ interface HighlightList { interface Filters { kind: "" | Kind; min: number; - pick: "all" | "open" | "keep" | "drop"; + pick: "best" | "all" | "open" | "keep" | "drop"; folder: string; sort: "score" | "clip"; } const PAGE = 200; +/** The most highlights a project can ask for (MAX_WANTED on the server). */ +const MAX_WANTED = 200; + +/** A typed number of highlights: null for none, NaN when it isn't one. */ +function wantedValue(text: string): number | null { + if (!text.trim()) return null; + const n = Number(text); + return Number.isInteger(n) && n >= 1 && n <= MAX_WANTED ? n : Number.NaN; +} + +/** Choosing the best is under way, or will be shortly. */ +const choosing = (l: HighlightList) => !!l.ranking && (l.ranking.status === "running" || (l.ranking.status === "waiting" && l.ranking.due)); /** Seconds → "M:SS.s": a position in a clip. */ function clock(t: number): string { @@ -154,13 +171,23 @@ export function HighlightsRoute({ id, navigate }: { id: string; navigate: (to: s load(false).catch((e) => setErr(String(e.message || e))); }, [load]); - // While clips are still being read, new picks arrive: look again now and then. + // A project that asked for a number of highlights opens on the best of them. + const opened = useRef(false); + useEffect(() => { + if (!list || opened.current) return; + opened.current = true; + if (list.wanted && filters.pick === "all") setFilters((f) => ({ ...f, pick: "best" })); + }, [list, filters.pick]); + + // While clips are still being read, new picks arrive, and the best are + // chosen once they are: look again now and then. const pending = list?.clips.pending ?? 0; + const busy = pending > 0 || (!!list && choosing(list)); useEffect(() => { - if (!pending || list?.paused) return; + if (!busy || list?.paused) return; const t = setInterval(() => load(true).catch(() => {}), 8000); return () => clearInterval(t); - }, [pending, list?.paused, load]); + }, [busy, list?.paused, load]); const more = async () => { if (nextOffset === null) return; @@ -177,6 +204,8 @@ export function HighlightsRoute({ id, navigate }: { id: string; navigate: (to: s try { const updated = await api.send("PATCH", `/api/projects/${id}/highlights/${h.id}`, { pick }); setItems((prev) => prev.map((x) => (x.id === h.id ? updated : x))); + // Dropping one of the best brings in the next: show it. + if (filters.pick === "best") load(true).catch(() => {}); setList((l) => { if (!l) return l; const counts = { ...l.counts }; @@ -194,7 +223,7 @@ export function HighlightsRoute({ id, navigate }: { id: string; navigate: (to: s setErr(String((e as Error).message || e)); } }, - [id], + [id, filters.pick, load], ); // A person's correction: who says it, as they know it ("Name, Title"). @@ -274,7 +303,7 @@ export function HighlightsRoute({ id, navigate }: { id: string; navigate: (to: s

Highlights{name && · {name}}

{list.asked && list.counts.total > 0 && (
- +
)} @@ -283,8 +312,18 @@ export function HighlightsRoute({ id, navigate }: { id: string; navigate: (to: s load(false)} /> ) : ( <> - load(false)} /> - + load(false)} + onWanted={(wanted) => { + // A new number: show its best. No number: everything. + if (wanted && filters.pick !== "best") setFilters({ ...filters, pick: "best" }); + else if (!wanted && filters.pick === "best") setFilters({ ...filters, pick: "all" }); + else load(false).catch(() => {}); + }} + /> + {err &&

{err}

} {items.length === 0 ? ( void }) { const [text, setText] = useState(brief); + const [count, setCount] = useState(""); const [busy, setBusy] = useState(false); const [err, setErr] = useState(""); + const wanted = wantedValue(count); const ask = async () => { setBusy(true); setErr(""); try { - await api.send("POST", `/api/projects/${projectId}/highlights`, { brief: text }); + await api.send("POST", `/api/projects/${projectId}/highlights`, { brief: text, ...(wanted ? { wanted } : {}) }); onAsked(); } catch (e) { setErr(String((e as Error).message || e)); @@ -363,15 +404,138 @@ function AskPanel({ projectId, brief, logged, onAsked }: { projectId: string; br placeholder="e.g. A 60 to 90 second sizzle of our presence at the event: soundbites from interviews and panels, b-roll of the booth, people and stage" className="field w-full mb-3" /> + {err &&

{err}

} - ); } -function Progress({ list, projectId, onAgain }: { list: HighlightList; projectId: string; onAgain: () => void }) { +/** How many highlights the video wants: a number, or empty for every pick worth a look. */ +function WantedField({ id, value, onChange, autoFocus }: { id: string; value: string; onChange: (v: string) => void; autoFocus?: boolean }) { + const bad = Number.isNaN(wantedValue(value)); + return ( +
+ +
+ onChange(e.target.value)} + placeholder="All" + aria-invalid={bad} + aria-describedby={`${id}-help`} + autoFocus={autoFocus} + className="field w-24 tabular-nums" + /> + + {bad + ? `A whole number from 1 to ${MAX_WANTED}, or empty.` + : "Empty: every pick worth a look. A number: every clip is still read, then the best that many are chosen from all the picks, for this brief."} + +
+
+ ); +} + +/** The best the project asked for: where choosing them stands, and changing how many. */ +function BestLine({ + list, + projectId, + onWanted, + editing, + setEditing, +}: { + list: HighlightList; + projectId: string; + onWanted: (wanted: number | null) => void; + editing: boolean; + setEditing: (v: boolean) => void; +}) { + const [count, setCount] = useState(list.wanted ? String(list.wanted) : ""); + useEffect(() => { + if (editing) setCount(list.wanted ? String(list.wanted) : ""); + }, [editing, list.wanted]); + const [busy, setBusy] = useState(false); + const [err, setErr] = useState(""); + const wanted = wantedValue(count); + const save = async (value: number | null) => { + setBusy(true); + setErr(""); + try { + await api.send("PUT", `/api/projects/${projectId}/highlights/wanted`, { wanted: value }); + setEditing(false); + onWanted(value); + } catch (e) { + setErr(String((e as Error).message || e)); + } finally { + setBusy(false); + } + }; + const n = list.wanted; + const r = list.ranking; + return ( + <> + {n && r && !editing && ( +

+ {r.status === "failed" ? ( + + Couldn't choose the best {n}: {r.error}{" "} + + + ) : list.clips.pending > 0 ? ( + `Best ${n}: chosen from all the picks once the clips are read.` + ) : choosing(list) ? ( + <> + + Choosing the best {n} of {plural(list.counts.total - list.counts.dropped, "pick", "picks")}… + + ) : r.status === "waiting" ? ( + `Best ${n}: chosen again as more footage comes in.` + ) : ( + `Best ${list.counts.best} chosen from ${plural(list.counts.total - list.counts.dropped, "pick", "picks")}, for the brief. Drop one and the next comes in.` + )} +

+ )} + {editing && ( +
+ + {err &&

{err}

} +
+ + +
+
+ )} + + ); +} + +function Progress({ + list, + projectId, + onAgain, + onWanted, +}: { + list: HighlightList; + projectId: string; + onAgain: () => void; + onWanted: (wanted: number | null) => void; +}) { const { clips, counts } = list; const [busy, setBusy] = useState(false); // Finding again is the moment to change what the picks are judged against. @@ -397,6 +561,7 @@ function Progress({ list, projectId, onAgain }: { list: HighlightList; projectId } }; const read = clips.done + clips.failed; + const [editingWanted, setEditingWanted] = useState(false); return (

@@ -415,6 +580,7 @@ function Progress({ list, projectId, onAgain }: { list: HighlightList; projectId

{list.paused &&

Paused: {list.paused}

} {list.brief && !editing &&

For: {list.brief}

} + {!editing && } {editing && (