Repository navigation
Re-derive dARSdS and correct the apparent mean open and shut times - #20
Merged
Merged
Conversation
exact_mean_open_shut_time was too large by exactly twice the dead time, and HJC_adjacent_mean_open_to_shut_time_pdf by once. The algebra in neither was wrong; the offset assumed for the dARSdS term was. That term evaluates to the mean apparent duration plus tres, not minus it, so the mean is recovered by subtracting tres rather than adding it. Checked against references that owe nothing to the implementation: for a two-state channel it now reproduces eqns (121) and (122) of Colquhoun & Hawkes (1995b) exactly, and for CH82 and the AChR mechanism it agrees with the mean of the asymptotic HJC pdf, tres + sum(area * tau), which reaches the same quantity by a different route through the library. At tres = 0 the apparent means become the ideal ones, as they must. The root cause is the convention of qmatlib.dARSdS, which has not been re-derived against CHS96 eqn (3.6). The correction is applied at each call site instead, where what is wanted is unambiguous. Anyone calling dARSdS directly still inherits the same trap. Adds HJC_adjacent_mean_open_to_shut_range_mean, the mean apparent open time for openings whose adjacent shut time falls in a given range -- what is superimposed on the observed conditional means in Fig. 6D of Colquhoun, Hatton & Hawkes (2003), and not the same thing as the continuous relationship, which cannot be compared with data directly because the data must be binned. It needs the apparent shut time density as a weight, so HJC_shut_time_pdf is exposed on its own rather than remaining a denominator inside the conditional mean. The finite-dead-time reference values in test_real_dtype.py move with the correction. They also move for a second, smaller reason recorded there: popen.EC50 was made to converge on the concentration rather than stopping as soon as Popen was within 0.001 of half maximal. Nothing about the real-dtype cast those tests guard has changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 2*tres offset patched at the call sites in the previous commit had a single cause, now found by doing the derivation the docstring only sketched. HJC give AR*(s) = [sI - QAA - QAF M(s) QFA]^-1 with M(s) = (I - exp(-(sI - QFF) tres)) (sI - QFF)^-1. Factorising as CHS96 eqn (3.6) does, AR*(s) = inv(VA(s)) inv(U(s)) with U(s) = sI - QAA and VA(s) = I - inv(U) QAF M(s) QFA, and differentiating the product of two inverses gives exactly the DARS formula already in the docstring. So the outer structure was right and only dVAds was in question. Differentiating M with N(s) = sI - QFF gives M'(0) = -tres expQFF inv(QFF) - SFF inv(QFF)^2, and hence dVAds = -inv(QAA) GAF SFF GFA - tres GAF expQFF GFA - GAF SFF inv(QFF) GFA Two of those three terms were coded correctly. The dead-time term was coded POSITIVE. That made dVAds too large by 2 tres GAF expQFF GFA, and every apparent mean duration built on it too long by 2 tres. With the sign fixed, the callers add tres rather than subtracting it, which is what the theory says: the density is written in u = t - tres and integrates to 1, since phi inv(VA) GAF expQFF u = 1. The numbers do not change -- all four independent checks in test_apparent_means.py still hold, and the 590 tests pass -- but the correction now lives where the error was instead of being cancelled three times downstream. Anyone calling dARSdS directly gets a correct answer for the first time. Three tests added. The one that matters differentiates AR*(s) numerically straight from its definition, using nothing else in the library, and compares with dARSdS at three dead times; a flipped sign fails it. The others pin the normalisation phi inv(VA) GAF expQFF u = 1 and the resulting mean = DARS term + tres. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 8, 2026
Merged
remislp
added a commit
that referenced
this pull request
Sep 11, 2026
Seven merged pull requests have been waiting since 1.1.0, and one of them is now blocking work elsewhere: HJCFIT's fitting layer is duck-typed on a mechanism and needs Rate.is_free, which exists here on master and not in the published 1.1.0. Its [fitting] extra pins scalcs>=1.1.0, which is too loose -- the integration tests fail on the released wheel with "'Rate' object has no attribute 'is_free'". A minor rather than a patch release, because it corrects computed values as well as adding API. #17 root bracketing for the asymptotic pdf. Counts were not recomputed when a search bound moved, widening happened once where it needed to repeat, and the upper bound was compared against the wrong state count. det[W(s)] is also unusable at large |s|, where it is of order 1e53 and changes sign hundreds of times in noise, so roots are now refined by bisecting the eigenvalue count where the determinant cannot be trusted. Roots are validated before being returned. #18 scsim re-exports burst segmentation from dcio; one implementation in the stack rather than two. #20 the dARSdS sign. exact_mean_open_shut_time was too large by exactly twice the dead time and the conditional means by once. This changes numbers a caller will have seen. #21 impose_resolution checked against RESHJC2.FOR interval for interval. #22 the simulator checked against Q-matrix theory for multi-state mechanisms, not only the two-state case where a reciprocal rate is the answer. #23 qmatlib.H inverted sI - QFF directly, which fails where the root search walks. The singularity is removable; only the factored form has a pole. #24 GPL-2.0-or-later rather than GPL-2.0-only, without which HJCFIT (GPL-3.0-or-later) could not depend on this at all. Both version files are bumped; the versions-agree gate added in #16 is what makes that hard to get wrong. Verified: the built wheel reports 1.2.0, carries its three data files, declares GPL-2.0-or-later, and Rate.is_free is present when installed from it into a clean environment. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
exact_mean_open_shut_timewas too large by exactly twice the dead time, andHJC_adjacent_mean_open_to_shut_time_pdfby once.The cause is one sign in
qmatlib.dARSdS.The derivation
HJC give the Laplace transform as
Factorising as CHS96 eqn (3.6) does, pulling out
(sI - QAA):Differentiating the product of two inverses reproduces exactly the
DARSformula already in the docstring, so the outer structure was never in question — onlydVAds.With
N(s) = sI - QFF,M'(0) = -tres expQFF inv(QFF) - SFF inv(QFF)^2, and thereforeTwo of those three terms were right. The dead-time term was coded positive. That made
dVAdstoo large by2 tres GAF expQFF GFA, and every apparent mean duration built on it too long by2 tres.What changes
With the sign fixed, the three call sites add
tresrather than subtracting it — which is what the theory says, because the density is written inu = t - tresand integrates to 1 (phi inv(VA) GAF expQFF u = 1, tested).The numbers do not change. The correction now lives where the error was, instead of being cancelled three times downstream — and anyone calling
dARSdSdirectly gets a correct answer for the first time.How it was checked
Against references that owe nothing to the implementation:
AR*(s), formed straight from its definition using nothing else in the library, at three dead times. A flipped sign fails it. This is the test that pins the root cause.tres + sum(area * tau)— a different route through the library.tres = 0, where the apparent means must become the ideal ones.Also in this PR
HJC_adjacent_mean_open_to_shut_range_mean— the mean apparent open time for openings whose adjacent shut time falls in a given range. This is what is superimposed on the observed conditional means in Fig. 6D of Colquhoun, Hatton & Hawkes (2003), and is not the continuous relationship, which cannot be compared with data directly because the data must be binned. It needs the apparent shut time density as a weight, soHJC_shut_time_pdfis exposed on its own.Test changes
The finite-dead-time reference values in
test_real_dtype.pymove with the correction, and for a second, smaller reason recorded there:popen.EC50now converges on the concentration rather than stopping as soon asPopenis within 0.001 of half maximal. Nothing about the real-dtype cast those tests guard has changed.tests/test_apparent_means.pyis new and carries the checks above.Verification
590 passed, 1 skipped.Found while reproducing Colquhoun, Hatton & Hawkes (2003), where the corrected means are needed for Fig. 6D; that reproduction's own step checks pass against this branch.
🤖 Generated with Claude Code