Skip to content

🆕 Add Virchow2 distilled model configs - #1131

Open
Jiaqi-Lv wants to merge 15 commits into
developfrom
Add-Virchow2-distilled-models
Open

Jiaqi-Lv wants to merge 15 commits into
developfrom
Add-Virchow2-distilled-models

Conversation

@Jiaqi-Lv

@Jiaqi-Lv Jiaqi-Lv commented Sep 4, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This pull request adds the SegFormer-B5 tissue segmentation models distilled from Virchow2, plus the inference and export fixes those models need.

The models come from Computationally Efficient Pathology Segmentation using Knowledge Distillation from Foundation Models.

New model configs

pretrained_model.yaml registers five checkpoints from TIACentre/Virchow2_Distilled_Segmentation_Models:

Model Dataset Classes Resolution Patch
segformerb5_distilled_PUMA_1/2/5 PUMA 6 (background, stroma, blood vessels, tumour, epidermis, necrosis) 1.14 mpp 224, stride 112
segformerb5_distilled_IGNITE_1 IGNITE 7 (unknown, background, tumour, stroma, necrosis, macrophages, rest) 0.5 mpp 448, stride 224
segformerb5_distilled_BEETLE_1 BEETLE 5 (unknown, other, non-invasive epithelium, invasive epithelium, necrosis) 1.0 mpp 448, stride 224

SegFormer inference

  • Segformer takes an optional class_dict.
  • Forward no longer relies on the Hugging Face segmentation head's hooks. Those hooks are shared across DataParallel replicas and feed the same feature map into every decoder projection. The model now runs each encoder stage directly and passes those hidden states to decode_head.
  • load_state_dict unwraps checkpoints nested under a model key.
  • postproc accepts a Dask array, runs the morphology on a computed array, and returns a Dask array so the engine can write the mask to Zarr.

QuPath export

Semantic-segmentation GeoJSON now uses contour hierarchy, so a nested class becomes a hole in the surrounding polygon. Classes with no pixels are skipped.

Example

Notebook 06-semantic-segmentation runs segformerb5_distilled_PUMA_1 on a whole-slide image, writes an annotation store, and shows the overlay in TIAViz.

@codecov

codecov Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.91%. Comparing base (c0bbb14) to head (c15d4fe).

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #1131   +/-   ##
========================================
  Coverage    99.91%   99.91%           
========================================
  Files          112      112           
  Lines        13454    13470   +16     
  Branches      1610     1614    +4     
========================================
+ Hits         13443    13459   +16     
  Misses           5        5           
  Partials         6        6           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@shaneahmed shaneahmed added this to the Release v2.1.4 milestone Sep 11, 2026
@shaneahmed shaneahmed added the enhancement New feature or request label Sep 11, 2026
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@Jiaqi-Lv
Jiaqi-Lv marked this pull request as ready for review October 15, 2026 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants