Skip to content

Check the simulator against Q-matrix theory for multi-state mechanisms - #22

Merged
remislp merged 1 commit into
masterfrom
simulator-vs-theory-tests
Sep 8, 2026
Merged

remislp merged 1 commit into
masterfrom
simulator-vs-theory-tests

Conversation

@remislp

@remislp remislp commented Sep 8, 2026

Copy link
Copy Markdown
Member

test_scsim.py checks the two-state CO mechanism, where the means are 1/alpha and 1/beta. That is the easy case: with one open and one shut state the dwell times are single exponentials, and a simulator can get them right while getting the multi-state case wrong.

These check CH82 and AChR_diamond, where the answer is a Q-matrix expression rather than a reciprocal rate:

  • the ideal means, phi_A (-Q_AA)^-1 u and phi_F (-Q_FF)^-1 u;
  • P(open) against the equilibrium occupancies;
  • and, after a dead time is imposed, the apparent means against scalcslib.exact_mean_open_shut_time.

The last one earns its keep

Nothing else ties the simulator to the missed-events code. test_apparent_means.py checks that code against closed forms and against itself by another route, but never against data. An error of two dead times in the apparent mean open time — exactly what was there until the dARSdS sign was corrected in #20 — is caught here even if every analytic cross-check had been adjusted to agree with it.

Verified by mutation: putting the error back (+ tres → - tres in exact_mean_open_shut_time) makes both mechanisms fail; reverting restores them.

How much power, measured rather than asserted

The sensitivity depends on the dead time as a fraction of the mean open time, so it differs by mechanism. At 400 000 intervals and tres = 25 µs:

mechanism one t_res two t_res actual disagreement
CH82 3.4 SE 6.7 SE 1.3 SE
AChR_diamond 5.6 SE 11.3 SE 1.0 SE

So at the 4 SE tolerance a two-dead-time error is caught on both, and a one-dead-time error on the AChR mechanism but not on CH82, whose open times are a hundred times the dead time.

test_the_apparent_open_time_check_has_power asserts the two-dead-time case everywhere and the one-dead-time case only where the measurement supports it. An earlier draft claimed one dead time everywhere and failed honestly at 100 000 intervals — which is how the numbers above came to be measured rather than guessed. Claiming more than that would make a passing suite mean less than it appears to.

Tolerances

Multiples of the measured standard error rather than fixed percentages, so the tests neither pass by being loose on a sharp quantity nor fail by being tight on a noisy one. Seeds are fixed, so a failure is a real disagreement rather than a bad draw.

Marked slow: 616 passed with them, 598 in 32 s without.

🤖 Generated with Claude Code

test_scsim.py checks the two-state CO mechanism, where the means are 1/alpha
and 1/beta. That is the easy case: with one open and one shut state the dwell
times are single exponentials, and a simulator can get them right while
getting the multi-state case wrong.

These check CH82 and AChR_diamond, where the answer is a Q-matrix expression
rather than a reciprocal rate: the ideal means phi_A (-Q_AA)^-1 u and
phi_F (-Q_FF)^-1 u, P(open) against the equilibrium occupancies, and -- after
a dead time is imposed -- the apparent means against
scalcslib.exact_mean_open_shut_time.

The last earns its keep. Nothing else ties the simulator to the missed-events
code: test_apparent_means.py checks that code against closed forms and against
itself by another route, but never against data. An error of two dead times in
the apparent mean open time -- exactly what was there until the dARSdS sign was
corrected -- is caught here even if every analytic cross-check had been
adjusted to agree with it. Verified by mutation: putting the error back makes
both mechanisms fail.

How much power the check has was measured rather than asserted, because it
depends on the dead time as a fraction of the mean open time. At 400000
intervals and tres = 25 us:

  mechanism      one tres   two tres   actual disagreement
  CH82            3.4 SE     6.7 SE      1.3 SE
  AChR_diamond    5.6 SE    11.3 SE      1.0 SE

So a two-dead-time error is caught on both, and a one-dead-time error on the
AChR mechanism but not on CH82, whose open times are a hundred times the dead
time. The test that guards this asserts the two-dead-time case everywhere and
the one-dead-time case only where the measurement supports it; claiming
otherwise would make a passing suite mean less than it appears to. An earlier
draft did claim it, and failed honestly at 100000 intervals, which is how the
numbers above came to be measured.

Tolerances are multiples of the measured standard error rather than fixed
percentages, so the tests neither pass by being loose on a sharp quantity nor
fail by being tight on a noisy one. Seeds are fixed.

Marked slow: 616 passed with them, 598 in 32 s without.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@remislp
remislp merged commit d7905ff into master Sep 8, 2026
9 checks passed
@remislp
remislp deleted the simulator-vs-theory-tests branch September 8, 2026 09:49
@remislp remislp mentioned this pull request Sep 11, 2026
remislp added a commit that referenced this pull request Sep 11, 2026
Seven merged pull requests have been waiting since 1.1.0, and one of them is
now blocking work elsewhere: HJCFIT's fitting layer is duck-typed on a
mechanism and needs Rate.is_free, which exists here on master and not in the
published 1.1.0. Its [fitting] extra pins scalcs>=1.1.0, which is too loose --
the integration tests fail on the released wheel with "'Rate' object has no
attribute 'is_free'".

A minor rather than a patch release, because it corrects computed values as
well as adding API.

  #17  root bracketing for the asymptotic pdf. Counts were not recomputed when
       a search bound moved, widening happened once where it needed to repeat,
       and the upper bound was compared against the wrong state count. det[W(s)]
       is also unusable at large |s|, where it is of order 1e53 and changes sign
       hundreds of times in noise, so roots are now refined by bisecting the
       eigenvalue count where the determinant cannot be trusted. Roots are
       validated before being returned.
  #18  scsim re-exports burst segmentation from dcio; one implementation in the
       stack rather than two.
  #20  the dARSdS sign. exact_mean_open_shut_time was too large by exactly
       twice the dead time and the conditional means by once. This changes
       numbers a caller will have seen.
  #21  impose_resolution checked against RESHJC2.FOR interval for interval.
  #22  the simulator checked against Q-matrix theory for multi-state
       mechanisms, not only the two-state case where a reciprocal rate is the
       answer.
  #23  qmatlib.H inverted sI - QFF directly, which fails where the root search
       walks. The singularity is removable; only the factored form has a pole.
  #24  GPL-2.0-or-later rather than GPL-2.0-only, without which HJCFIT
       (GPL-3.0-or-later) could not depend on this at all.

Both version files are bumped; the versions-agree gate added in #16 is what
makes that hard to get wrong. Verified: the built wheel reports 1.2.0, carries
its three data files, declares GPL-2.0-or-later, and Rate.is_free is present
when installed from it into a clean environment.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant