Autocut: jump-cut silences, fillers, and stutters (inspired by CapCut) - #36
Open
felipebasurto wants to merge 15 commits into
Open
Autocut: jump-cut silences, fillers, and stutters (inspired by CapCut)#36felipebasurto wants to merge 15 commits into
felipebasurto wants to merge 15 commits into
Conversation
Compose waveform silence detection with transcript word timings to propose keep-ranges for jump-cut edits. Drops silences, stutters, and common filler words without re-encoding. Optional --jsx emits a <sequence> of trimmed <video> clips. Co-authored-by: Felipe Basurto <felipebasurto@users.noreply.github.com>
- Restrict filler drops to vocal pauses and safe phrases only - Emit probe width/height in JSX; use <audio> for audio assets - Continue with silence-only cuts when transcribe finds no speech - Add node:test coverage for computeAutocut and formatAutocutJsx - Document absolute source-second clock alignment with waveform Co-authored-by: Felipe Basurto <felipebasurto@users.noreply.github.com>
When cloud STT fails without auth, resolveTranscript runs openai-whisper or whisper-cli on local file paths. Adds --transcript escape hatch and documents offline install paths without requiring sign-in. Co-authored-by: Felipe Basurto <felipebasurto@users.noreply.github.com>
Move autocut analysis into @diffusionstudio/runtime so CLI and web share the same keep-range logic. Wire a one-click Autocut pill on the canvas ActionBar when a single video or audio clip is selected: compose waveform silences and transcription in the renderer, then replace the clip with a sequence row of trimmed copies. Desktop falls back to local whisper via MEDIA_TRANSCRIBE_LOCAL when cloud STT is unavailable. Co-authored-by: Felipe Basurto <felipebasurto@users.noreply.github.com>
Show Autocut when exactly one video or audio node is selected, matching Add audio/Upscale tag detection instead of requiring a library AssetId. Resolve media via src through AssetLibrary.resolve for waveform and STT. Use asset.source for local whisper IPC so absolute paths work. Co-authored-by: Felipe Basurto <felipebasurto@users.noreply.github.com>
|
@cursoragent is attempting to deploy a commit to the Diffusion Studio Team on Vercel. A member of the Team first needs to authorize it. |
felipebasurto
marked this pull request as ready for review
August 29, 2026 15:26
Co-authored-by: Felipe Basurto <felipebasurto@users.noreply.github.com>
Detect Autocut targets via findGeometryAsset (VIDEO/SEQUENCE/AUDIO) instead of authored video/audio tags only, so library drops show the ActionBar pill. Apply clones the authored Rect subtree with per-span timing for geometry clips; direct video/audio JSX clips keep the existing path. Tests run via tsx for Node 20 compatibility. Co-authored-by: Felipe Basurto <felipebasurto@users.noreply.github.com>
Move jump-cut UI from ActionBar to Object menu → AI actions → Remove silences. Keep shared computeAutocut in runtime, CLI command, reference doc, and minimal web engine glue. Drop local whisper stack, ActionBar pill, mock docs, and check-script churn. Co-authored-by: Felipe Basurto <felipebasurto@users.noreply.github.com>
Co-authored-by: Felipe Basurto <felipebasurto@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Inspired by CapCut's Remove Silence and Remove filler words — the jump-cut cleanup, not the full transcript editor.
Drop a talking-head on the timeline, click Autocut, get one row of clips with silences, fillers (
um/uh/eh,you know,o sea, …) and immediate stutters gone. No re-encode:sourceIn/sourceOuton a<sequence>.Agents already had the same apply:
UI
Select the clip on the timeline. Autocut sits on the floating ActionBar under the canvas (same pill as Auto-Captions / Add audio). One click, no dialog. Works on library assets and path-based
<video src>/<audio src>clips. Defaults: silence ≥ 0.4s, 50ms pad,en+esfillers.Select the clip → Autocut
Before / after
Same talking-head, keep-ranges applied as a sequence (what the button writes).
Composer smoke on a messy first-vlog (50s source): 23 keep ranges, 32.5s, 20 silences + 1
um+ 2 stutters. Runtime tests forcomputeAutocut/planAutocutTimeline: 6/6.What it is not
Not Descript-style transcript editing. CapCut has that as a separate layout. This PR only does the cleanup pass: pauses, repeats, filler vocalizations.
How to try
dapi media autocut <path> --jsx(cloud STT when signed in; localwhisper/whisper-cliwhen not).