Repository navigation
GitHub ingest: skip slow or oversized attachment downloads instead of dead-lettering - #1392
Conversation
… dead-lettering A PR body attachment that took longer than the 8 s GitHub API deadline to download threw a TimeoutError, so the queue retried the whole webhook event six times in two minutes and dead-lettered it, dropping the adoption and comment sync for that PR too (fintentional-monorepo#7799, Nov 5). Attachment downloads now get a 30 s deadline, a declared Content-Length over the workspace ceiling is skipped as too_large before the body is read, and a download that still times out is a permanent asset_timeout skip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeLbkfu5FcEeveF5EnSXvd
|
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one): 2
🚫 Excluded labels (none allowed): 1
Please check the settings in the CodeRabbit UI or the
 |
Requested by Zach · project thread
Why
Two
pull_requestedited deliveries for a private repo's PR (Nov 5, ~09:01 UTC) dead-lettered. In the Worker logs, every one of the 7 attempts per event hitTimeoutError: The operation was aborted due to timeoutabout 8 s into a fetch. That's thegithubFetchdeadline. Other events in the same batches, including GitHub API calls for other installations, went through fine. The repo's earlier PRs ingest normally, so nothing is wrong with the installation. The failing call is the attachment download infetchAndStore: the 8 s API deadline also covers reading the body, and a large screen recording in the PR body can't finish in that time.Because the timeout throws, the queue retried the whole event 6 times in about 2 minutes (no retry delay), and every retry timed out the same way. The event then dead-lettered, and the PR's adoption and managed-comment sync were lost with it.
What
githubFetchtakes an optionaltimeoutMs.Content-Lengthis over the workspace's largest ceiling, the asset is skipped astoo_largebefore the body is read.asset_timeoutskip. It no longer throws, so the rest of the event (other assets, adoption, comment sync) still runs. Skips aren't ledgered, so a later edit tries the asset again.Test plan
pnpm --filter @uploads/api exec vitest run src/github-ingest.test.ts src/github-app.test.ts: 85 passed, including two new cases (timeout becomes a skip; an oversizedContent-Lengthis skipped without a put).pnpm --filter @uploads/api typecheckpasses.Generated by Claude Code