Skip to content

Use each bin's width in dndm_from_sample - #427

Merged
steven-murray merged 3 commits into
halomod:mainfrom
akashdeshmukh-git:fix-dndm-uneven-bins
Oct 26, 2026
Merged

steven-murray merged 3 commits into
halomod:mainfrom
akashdeshmukh-git:fix-dndm-uneven-bins

Conversation

@akashdeshmukh-git

@akashdeshmukh-git akashdeshmukh-git commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

dndm_from_sample() took the width of every bin from the first one, so dn/dm was wrong in all other bins when bins were given as non-uniform edges, which the bins parameter allows.

Summary

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance (refactoring, CI, dependencies, etc.)

Linked issues

Checklist

I have

  • Added or updated tests covering this change.
  • Updated docstrings and/or docs where relevant.

Summary by Sourcery

Fix dndm_from_sample to calculate mass functions correctly for non-uniform bins.

Bug Fixes:

  • Correct dn/dm normalization for samples binned with non-uniform mass-bin edges by using each bin’s width.
  • Add regression coverage for uneven-bin mass functions.

dndm_from_sample() took the width of every bin from the first one, so
dn/dm was wrong in all other bins when bins were given as non-uniform
edges, which the bins parameter allows.
@sourcery-ai

sourcery-ai Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Fix dndm_from_sample so each bin is normalized by its own width in log10 mass, and add a regression test confirming correct dn/dm values for non-uniform bin edges.

Flow diagram for per-bin dndm normalization

flowchart LR
    M["Sample masses m"] --> H["np.histogram(np.log10(m), bins, weights=nm)"]
    H --> E["Bin edges"]
    E --> D["np.diff(edges)"]
    E --> C["Bin centres"]
    D --> N["Normalize each bin by its own width"]
    C --> N
    N --> R["dndm_from_sample result"]
Loading

File-Level Changes

Change Details Files
Normalize each histogram bin using its own logarithmic width.
  • Replace the single width derived from the first two bin centres with the per-bin widths from adjacent edges.
  • Preserve the existing dn/dm conversion while correcting results for non-uniform bins.
src/hmf/helpers/sample.py
Add regression coverage for non-uniform bin edges.
  • Generate a uniformly distributed log-mass sample with a known analytic dn/dm.
  • Verify the calculated values across uneven bins using a tight numerical tolerance.
tests/test_sample.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="src/hmf/helpers/sample.py" line_range="108" />
<code_context>
     hist, edges = np.histogram(np.log10(m), bins, weights=nm)
     centres = (edges[1:] + edges[:-1]) / 2
-    dx = centres[1] - centres[0]
+    dx = np.diff(edges)
     hist = hist.astype("float") / (10**centres * float(V) * dx * np.log(10))

</code_context>
<issue_to_address>
**Concurrent bin changes skew normalization**

When another thread mutates the ndarray passed as `bins` between the `centres` calculation and `np.diff(edges)`, `np.histogram` returns the supplied edge array, so `dx` can read newly mutated widths while `hist` and `centres` reflect the earlier edges, producing a misnormalized result.

Copy caller-provided bin edges before passing them to `np.histogram`, and use that snapshot throughout the calculation.
</issue_to_address>

### Comment 2
<location path="tests/test_sample.py" line_range="71" />
<code_context>
+    centres, hist = dndm_from_sample(m, volume, bins=bins)
+
+    expected = n / (volume * 3 * 10**centres * np.log(10))
+    assert np.allclose(hist, expected, rtol=1e-10)
</code_context>
<issue_to_address>
**Uneven-bin test misses the bug**

When the old equal-width normalization is used, `np.allclose` uses its default `atol=1e-8`, much larger than the expected histogram values in the higher-mass bins (about `1e-10`). The old equal-width normalization therefore still satisfies this assertion, so the regression test passes with the bug present.

Set an explicit `atol` small enough for these values (or zero) so the test detects the incorrect bin normalization.
</issue_to_address>

Sourcery assessment

Approval pending. 2 findings to address first.

Blocking findings: src/hmf/helpers/sample.py:108, tests/test_sample.py:71


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread src/hmf/helpers/sample.py
hist, edges = np.histogram(np.log10(m), bins, weights=nm)
centres = (edges[1:] + edges[:-1]) / 2
dx = centres[1] - centres[0]
dx = np.diff(edges)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Concurrent bin changes skew normalization

When another thread mutates the ndarray passed as bins between the centres calculation and np.diff(edges), np.histogram returns the supplied edge array, so dx can read newly mutated widths while hist and centres reflect the earlier edges, producing a misnormalized result.

Copy caller-provided bin edges before passing them to np.histogram, and use that snapshot throughout the calculation.

Prompt for AI agents
In `src/hmf/helpers/sample.py` at line 108:

**Concurrent bin changes skew normalization**

When another thread mutates the ndarray passed as `bins` between the `centres` calculation and `np.diff(edges)`, `np.histogram` returns the supplied edge array, so `dx` can read newly mutated widths while `hist` and `centres` reflect the earlier edges, producing a misnormalized result.

Copy caller-provided bin edges before passing them to `np.histogram`, and use that snapshot throughout the calculation.

Comment thread tests/test_sample.py Resolved
@codecov

codecov Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.60%. Comparing base (3b61ccd) to head (7225984).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #427   +/-   ##
=======================================
  Coverage   99.60%   99.60%           
=======================================
  Files          62       62           
  Lines        7819     7819           
=======================================
  Hits         7788     7788           
  Misses         31       31           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

np.allclose defaults to atol=1e-8, larger than the expected dn/dm in the
upper bins (about 1e-10), so those bins were not effectively checked.

@akashdeshmukh-git akashdeshmukh-git left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tolerance: fixed in 200473c with an explicit atol=0. Thread safety: not applicable. The function only uses the edges array returned by np.histogram, never bins after that call.

@steven-murray steven-murray left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @akashdeshmukh-git !

@steven-murray
steven-murray merged commit 8e36e2a into halomod:main Oct 26, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants