Skip to content

Isolate RNG in PowerNet battery energy distribution - #986

Closed
GetParanoid wants to merge 51 commits into
rwmt:masterfrom
GetParanoid:fix/powernet-battery-shuffle-desync
Closed

Isolate RNG in PowerNet battery energy distribution#986
GetParanoid wants to merge 51 commits into
rwmt:masterfrom
GetParanoid:fix/powernet-battery-shuffle-desync

Conversation

@GetParanoid

Copy link
Copy Markdown

Problem

PowerNet.DistributeEnergyAmongBatteries shuffles the net's battery list (batteriesShuffled.Shuffle()), consuming calls from the map's shared RNG.

Whether it runs on a given tick is gated by a float comparison in PowerNet.ChangeStoredEnergy (extra > 0f), fed by CurrentEnergyGainRate() / CurrentStoredEnergy() summing floats across every power component and battery on the net. Those sums drift slightly between machines, so the comparison can flip a tick earlier on one peer than another: one peer runs the shuffle on a tick where the other doesn't, Rand.iterations ends the tick at a different value on each, and the map desyncs with Wrong random state on map 0.

Evidence

From a 2-player session (MP 0.11.5, RimWorld 1.6.4871, no async time, single map, no mods patching PowerNet): the client's and host's recorded Rand stack traces match call for call — same ticks, same entities, same states — through tick 6781265, then diverge.

Client — stays on tick 6781265 and makes 7 RNG calls the host never makes, all from the same stack:

Rand.RangeInclusive -> GenList.Shuffle -> PowerNet.DistributeEnergyAmongBatteries
  -> PowerNet.ChangeStoredEnergy -> PowerNet.PowerNetTick -> Map.MapPostTick

Host — makes none of those calls, advancing straight from tick 6781265 to 6781266 (an unrelated ChildcareUtility.ShouldWakeUpToAutofeedUrgent check).

The same signature recurred 20+ times across the session, typically every few thousand ticks, and reappeared after each automatic rejoin. DistributeEnergyAmongBatteries shows up as a real stack frame in the traces, so it isn't being inlined.

Happy to attach the full desync logs if that's useful.

Fix

Push/pop the RNG state around the method while in MP, matching the approach in #968.

This targets exactly what the desync check compares. AsyncTimeComp.Tick records randState once per map tick in its finally, and TryAddMapRandomState stores (uint)(state >> 32) — the high half of Rand.StateCompressed, i.e. Rand.iterations. So ClientSyncOpinion.CheckForDesync's map comparison is effectively "did both peers make the same number of RNG calls this tick?"

Rand.PushState() saves (seed, iterations) and PopState() restores both, so whatever the shuffle consumes is discarded before the tick boundary is recorded. A peer that runs the method and a peer that doesn't record the identical value.

Why isolating the RNG is safe here

The shuffled order cannot affect the outcome, so discarding the RNG it draws changes nothing:

  1. Every battery in a pass receives the same amount — either the smallest AmountCanAccept across the list, or an even share of the energy remaining. Never a per-position amount.
  2. AmountCanAccept depends only on the battery itself, never on the others, so no battery's share is influenced by what was processed before it.
  3. The removal predicate is position-independent (amountCanAccept <= 0f || amountCanAccept == num2), so the same batteries drop out of the list under any ordering.
  4. energy is decremented by the same value the same number of times, so the float arithmetic is order-independent too.

Every permutation therefore leaves the batteries holding identical energy.

Two further notes:

  • PushState() with no seed continues from the current state rather than reseeding, so in-sync peers still produce the same shuffle as each other. Behaviour is unchanged beyond the isolation.
  • Singleplayer is untouched — the patch returns early when Multiplayer.Client == null.

Scope

PowerNet.PowerNetTick has two other float-gated RNG consumers — partsWantingPowerOn.RandomElement() and potentialShutdownParts.RandomElement() — reachable depending on how num2 + num >= -1E-07f lands. Both are left alone: their results decide which building powers on or shuts down, so isolating that RNG would make peers pick different buildings and diverge for real. The battery shuffle is the one consumer in this path whose result provably doesn't affect the outcome, which is what makes it safe to isolate. The other two also sit behind tick-interval and non-empty-list conditions, so they fire far more rarely than the shuffle, which runs whenever a net has surplus energy.

This doesn't eliminate the underlying float drift — that's inherent to float math across machines, and the desync check doesn't look at it. It removes the drift's ability to become a random-state mismatch.

Testing

  • Compiles clean against dev, no new warnings.
  • The diagnosis is confirmed in-game. Patching DistributeEnergyAmongBatteries no longer draws from the shared RNG, took a 2-player colony from desyncing and rejoining every few thousand ticks to roughly 3 hours with no desync at all.

mibac138 and others added 30 commits April 30, 2026 01:51
* Add debug action to show mod compat when playing

Requested by a mod tester

* Allow clicking a mod name in mod compat dialog to open the mod's workshop page
To avoid a player having their own selection highlighted as if it was
another player's selection
Shows XML mods as a 4 to save on the limited space and avoid needlessly complicating the logic
* Use TargetMethods on the check to prevent missing patch

---------

Co-authored-by: Sakura-TA <Sakura_TA@163.com>
Co-authored-by: Sakura-TA <Sakura_TA@163.com>
* Detect registering the same SyncMethod multiple times

* Fix broken Hediff_Pregnant sync methods

* Remove duplicate SyncMethods

- RenamableLabel is handled at the end of SyncMethods for all IRenamables
- GameComponent_PsychicRitualManager.ClearAllCooldowns is handled in SyncDelegates near other Anomaly stuff
Bumps [Languages](https://github.com/rwmt/Multiplayer-Locale) from `8bfffb7` to `a8513b0`.
- [Commits](rwmt/Multiplayer-Locale@8bfffb7...a8513b0)

---
updated-dependencies:
- dependency-name: Languages
  dependency-version: a8513b0bc1ff212751c498e643241beeca23b953
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* ci: add standalone server zip to continuous release

* Potential fix for pull request finding

Avoid double build

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* ci: add standalone server download and bootstrap instructions to release notes

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Split standalone save trigger foundation

* Unify standalone save path and add Days autosave support

* Add standalone snapshot persistence

* Add safe defaults to snapshot state structs

* Use File.Replace in SaveGameToFile_Overwrite and eliminate double snapshot

* Enforce standalone async time and control fixes

* Reset MultiplayerServer.instance in TearDown

* Use int.TryParse in SeedFromSaveZip

* Clean up standalone prepublish maintenance

* Correct misleading blocked log text in designator patches

* Pass sourcePlayer to standalone join point creation for IssuedBySelf gate

* Fix: restore hosted SendGameData guard, remove redundant ofPlayer assignment

- CreateJoinPointAndSendIfHost: restore LocalServer/arbiter guard for hosted
  mode so only host/arbiter uploads world data (was accidentally ungated)
- Standalone path kept separate with ConnectedToStandaloneServer gate
- Remove redundant ofPlayer assignment in ChangeRealPlayerFaction
  (FactionContext.Set already does the same thing on the next line)

* Fix test failures: handle KeepAlive in test states and disable auto join point

- Add no-op HandleKeepAlive to TestJoiningState and TestLoadingKeepAliveState
  to prevent crash when server sends KeepAlive during async handshake
- Disable autoJoinPoint in test server settings since the test server has
  no game simulation to process CreateJoinPoint commands

* Update Source/Common/WorldData.cs

Co-authored-by: Michael <5672750+mibac138@users.noreply.github.com>

* Restrict world-travel join point trigger to streaming mode

* Remove unrelated client-side changes from PR scope

* Fix standalone join point source and trim snapshot metadata

* Remove unused JoinPointRequestReason.Unknown; remove unnecessary isStandaloneServer reset in Stop()

* Fix post-rebase build errors: add missing using, toml preview fields, Tab.Preview

* Align bootstrap configurator with upstream state handling

* Apply bootstrap settings upload review suggestion

---------

Co-authored-by: Michael <5672750+mibac138@users.noreply.github.com>
* fix(Determinism): force single-batch FastTileFinder.Query in MP to prevent quest site tile divergence

* fix(FactionContext): handle transporters and gravship map gen faction context

* fix(FactionContext): push gravship faction context during ArriveNewMap

* Count building's ownership when trading

---------

Co-authored-by: Sakura-TA <Sakura_TA@163.com>
* fix(Determinism): force single-batch FastTileFinder.Query in MP to prevent quest site tile divergence

* fix(FactionContext): handle transporters and gravship map gen faction context

* fix(FactionContext): push gravship faction context during ArriveNewMap

* use IsPlayer check when context is Spectator

* Removed unused TargetMethod from Map_IsPlayerHome_Spectator_Patch.

---------

Co-authored-by: Sakura-TA <Sakura_TA@163.com>
Co-authored-by: Meru <notfood@users.noreply.github.com>
* Typed ClientInitDataPacket.Mods

* Refactor RemoteData

Remove IConnector from it in favor of passing it around separately, and add a factory method to create it from a packet
* ci: add standalone server zip to continuous release

* Potential fix for pull request finding

Avoid double build

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* ci: package server beta artifacts for Windows and Linux

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Fix render burn

* Address SaveAndReload review
* Use shared entry transition for bootstrap reconnect

* Add missing System import for Rejoiner helper
Bumps [Languages](https://github.com/rwmt/Multiplayer-Locale) from `a8513b0` to `90a86c7`.
- [Commits](rwmt/Multiplayer-Locale@a8513b0...90a86c7)

---
updated-dependencies:
- dependency-name: Languages
  dependency-version: 90a86c76f8f15479069ff92cbaa29cf51f861f1d
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Fix join point stuck state on dedicated server

On a dedicated server no player is ever IsHost (hostUsername is never
set), so AbortJoinPointCreation was never called on disconnect. When a
client drops mid-join-point creation the server was left stuck in
CreatingJoinPoint forever, blocking all subsequent connections at
WaitJoinPoint().

Fix: also abort when no joined players remain after a disconnect.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update log message

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Meru <notfood@users.noreply.github.com>
NemuruYama and others added 21 commits June 1, 2026 11:28
* Prototype for no error on create join-point

* Optimize join-point reload redraw

* Clean up join-point reload optimization

* Expand reload optimization tests

* Remove net48 tests
)

* Convert chat commands into source generated registry command

* Fix generated enum defaults for chat command args

* Validate chat rest command arguments

* Validate chat command parser accessibility

* Validate chat command construction

* Validate generated chat command names

* Make chat command dispatch case-insensitive

* Extra commands to test the system

* Move chat command permissions onto commands

* Add usable-command help preference

* Parse chat command player arguments

* Request client rejoin for resync command

* Preserve raw chat command usage text

* Render raw chat messages without rich text

* Update protocol version

* Set default value for helpOnlyUsableCommands to true

* Tokenize quoted chat command arguments

* Infer rest parsing for single chat command arguments

* Clean up generated code

* Small code cleanup

* Added small pagination to mods command

* Temporarily switch to my fork of languages

* Fix chat window breaking with wrapped text
* Fix standalone join-point creation on join

* Address review feedback on standalone join fix
* Update for Rimworld 1.6.4850

"GenConstruct.CanPlaceBlueprintAt" from stable version now just calls the new "GenConstruct.CanPlaceBlueprintAt_NewTemp", which bugs existing MP transplier on load.
"GenConstruct.CanPlaceBlueprintAt_NewTemp" Mostly repeats already existing method, so I just changed targets of a patch. Also updated packages for the latest Harmony and (obviously) latest unstable patch.

---------

Co-authored-by: Meru <notfood@users.noreply.github.com>
Bumps [Languages](https://github.com/rwmt/Multiplayer-Locale) from `407942a` to `0a6b746`.
- [Commits](rwmt/Multiplayer-Locale@407942a...0a6b746)

---
updated-dependencies:
- dependency-name: Languages
  dependency-version: 0a6b746ca9b2d05a0cc5770f5348ec077f980deb
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Meru <notfood@users.noreply.github.com>
…#950)

This is implemented only for x86 and using it on other arches breaks the game tick.
StopMultiplayerAndClearAllWindows nulls Multiplayer.session, so reading
session.connector afterwards crashed before JoinDataWindow could open.
Capture the connector while the session is still alive.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…ion (rwmt#957)

When a group of drafted pawns is given a move order, PawnGotoAction runs
once per pawn on the client that issued the order. We already sync its
TryTakeOrderedJob call, but it has another path: when a pawn is already
standing on gotoLoc and its current job is Goto, it calls
Pawn_JobTracker.EndCurrentJob directly. That call isn't synced, so the
pawn stops only for the player who issued the order while it keeps walking
for everyone else, causing a desync.

Redirect that EndCurrentJob call to a synced wrapper through the existing
DraftedMove_GotoFeedbackPatch transpiler, the same way the
TryTakeOrderedJob call in the same method is already handled.

Co-authored-by: chimook <chimook.lee@fluentt.com>
)

Building the gravship launch confirmation dialog decides which pawns can
board in Dialog_BeginRitual's constructor: CreateRitualRoleAssignments ->
RitualRoleAssignments.PawnNotAssignableReason ->
RitualBehaviorWorker_GravshipLaunch.PawnCanFillRole -> CanReachGravship ->
GravshipUtility.TryFindSpotOnGravship -> Region.RandomCell, which consumes
RNG. Building the dialog is UI work opened only by the issuing peer (the
currentExecutingCmdIssuedBySelf gate in CancelDialogBeginRitual), so that
RNG runs on just one peer and advances the shared stream on that peer only.
Spamming the launch button repeats it and the divergence accumulates into a
desync ("Random state from commands doesn't match").

Wrap CanReachGravship in Rand.PushState/PopState, matching the existing
SeedPreceptComp_UnwillingToDo_Chance isolation for RNG that "can be called in
interface". CanReachGravship returns a bool that does not depend on the RNG
(it only searches for any allowed cell), and the real boarding calls
TryFindSpotOnGravship outside this check, so isolating here leaves both the
check result and determinism intact.

Co-authored-by: chimook <chimook.lee@fluentt.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@GetParanoid GetParanoid closed this Sep 9, 2026
@GetParanoid

Copy link
Copy Markdown
Author

Merged into wrong branch, oops.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.