Repository navigation
RAD-16862 Stop stale Android wakes from starving the upload queue - #57
Merged
Merged
Conversation
A parked entry got a wake at its expiry, which is years ahead for a long-lived request. Unparking it left that wake in place. Each delayed WorkManager row holds a JobScheduler slot, and the in-process scheduler reads only the 200 oldest enqueued rows. Enough stale wakes filled both, so no queued work started. A parked entry now gets an expiry wake only when the expiry is within 24 hours. scheduleRun() cancels a wake that is no longer needed, so the boot sweep frees the slots on phones that are already stuck. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 28, 2026
arnonate
marked this pull request as ready for review
October 28, 2026 16:55
Merged
arnonate
added a commit
that referenced
this pull request
Oct 28, 2026
Jira: [RAD-16862](https://openspaceai.atlassian.net/browse/RAD-16862) Bumps the version to 11.0.2 and adds the CHANGELOG note for #57, which had none. Master is protected, so the release commit needs a PR this time. 🤖 Generated with [Claude Code](https://claude.com/claude-code) [RAD-16862]: https://openspaceai.atlassian.net/browse/RAD-16862?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
arnonate
added a commit
that referenced
this pull request
Oct 29, 2026
…nd a boot snapshot of WorkManager runs (#58) Jira: [RAD-17288](https://openspaceai.atlassian.net/browse/RAD-17288), follow-up to [RAD-16862](https://openspaceai.atlassian.net/browse/RAD-16862) On Android, the library's warnings and errors only reach logcat. When a customer's queue stalled for six days, support's exported logs couldn't show why. A stuck phone gave us no WorkManager state, and we had to infer stale wakes from timestamps. This adds `takeDiagnostics()`. Android writes each warning and error to a small file, keeping the newest 50. That includes runs in the background with no JS. Each time the module starts, which is once per launch plus once per JS reload, it also writes one `info` record. It describes the entries the boot sweep schedules, as the sweep found them, and every unfinished WorkManager run of the library before the sweep cleans up. That covers how many runs there are by state, how many are delayed wakes, how many of those are due more than 24 hours out, the highest run attempt count, and how many queued or running entries had no run at all. JS reads and clears the file with `takeDiagnostics()`. iOS returns an empty list for now. ## What to look at - `DiagnosticLog.kt`: the file format, the cap, and how failures are swallowed. - `QueueController.sweep()`: the snapshot is read before #57's wake cleanup, so it shows what the earlier process left. It keeps each entry as found, so an entry that was running when its process died still counts as running with its old `updatedAt`. ## Decisions and trade-offs - **A file, not a live event.** A headless worker has no JS, and the boot sweep runs before JS subscribes, so a live event would lose both. - **One WorkManager query per launch.** `getWorkInfosByTag(WORK_TAG)` reads every library row at once. A query per entry would run hundreds of times on the queue's only thread. - **Wakes are told apart by their initial delay.** WorkInfo doesn't expose the unique work name, and wakes and main chains share their tags. A wake due already has no delay and counts as a main-chain run. - **Drain on read.** Each call clears the log, so a consumer that fails to report the records loses them. This keeps the API to one call. - **The expected "no event emitter" warning stays in logcat only.** It fires once per dropped event, so persisting it could push the snapshot and real errors out of the 50-record cap. - **A snapshot on every module start with queued entries, even healthy ones.** The consumer decides what is worth reporting, for example when `delayedPastHorizon` is above 0 or `oldestUpdatedAt` is old. ## Test Plan - `./gradlew :react-native-background-upload:testDebugUnitTest`: 322 tests, 0 failures. New tests cover the log's cap, drain, corrupt-file handling, and a fresh instance reading what the last one left, Diag routing, and the snapshot's fields, including stale wakes with no entry left. - `yarn typecheck`, `yarn test` (192 pass) and `yarn lint:ci` pass. - I didn't build the iOS example app. The iOS change is a stub that resolves an empty list. 🤖 Generated with [Claude Code](https://claude.com/claude-code) [RAD-16862]: https://openspaceai.atlassian.net/browse/RAD-16862?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [RAD-17288]: https://openspaceai.atlassian.net/browse/RAD-17288?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Android uploads can stop for good. A request that gets a 401 parks, and the library schedules a wake at its expiry. Diana's requests expire in 10 years. A fresh token unparks the request, but the wake stays.
Each delayed WorkManager row holds one of the app's 20 JobScheduler slots until it runs. WorkManager's in-process scheduler reads only the 200 oldest enqueued rows. Enough stale wakes fill both, and then no queued work starts. This matches RAD-16862: 534 entries, all
queued, with no activity for hours on a working network.Before: a parked entry always gets a wake at its expiry, and the wake stays after the entry unparks.
After: a parked entry gets a wake only when it expires within 24 hours. When an entry is scheduled again, the library cancels any wake it no longer needs. The boot sweep does this too, so phones that are stuck now recover on the first launch after the update.
Evidence from the stuck phone
queuedat the same millisecond. That was Oct 21, 09:29:04.333 UTC. After that, no row was attempted again. Rows added the next day were never attempted at all.updateHeaders()moves rows that are not paused back toqueuedwith one timestamp. It does that only to parked rows. So the rows were parked on a 401, each with a wake 10 years ahead.What to look at
QueueController.scheduleRun(): the two places that cancel a stale wake.WorkManagerScheduler.cancelWake(): it checks for an unfinished wake before it cancels.Decisions and trade-offs
expiredat the first launch after its expiry, not exactly on time.cancelWake()reads WorkManager first. A WorkManager cancel reschedules all pending work even when the name has no rows. The boot sweep calls this for every queued entry.release()are unchanged. They run within 2 hours, so they slow the queue but do not stop it.Test Plan
./gradlew :react-native-background-upload:testDebugUnitTestfromexample/RNBGUExample/android: 311 tests, 0 failures.tscandjest(190 tests) pass.What's required for testing (prerequisites)?
An Android phone on 11.0.1 with a few hundred queued uploads that parked on a 401.
What are the steps to reproduce (after prerequisites)?
Compatibility
iOS uses URLSession, not WorkManager, so this problem does not apply there.
Checklist
README.md🤖 Generated with Claude Code