MATS — Morphometric Analysis Toolbox for Segmentation

Measure leaf area, length, and width in real-world units from a photo of leaves laid on a printed calibration template.

MATS detects four fiducial markers with RF-DETR, corrects perspective, segments the leaf with Otsu thresholding by default or optional BiRefNet, and writes measurements to a CSV.

Companion code for the manuscript (target journal: Plant Phenomics). BiRefNet is optional and runs entirely from a locally installed checkpoint (see Model weights).

Table of contents

• Installation
• Model weights
• Running MATS
  • Using the app
  • Using the command line
• Outputs
• Running on a compute cluster
• How it works
• Troubleshooting
• Reproducibility and citing
• License

---

Installation

MATS requires Python 3.10–3.13 (3.13 recommended) and Git LFS. Apart from Git LFS, the recommended pip installation method pulls all required packages from wheels with no other system libraries and no conda required.

Starting on a computer with no Python or Git? Follow docs/install.md. It covers macOS, Windows, and Linux step by step, including computers without administrator rights, and lists the common error messages with their fixes. The python3 built into macOS is 3.9, which is too old.

Step 1 — install Git LFS before cloning

The RF-DETR marker checkpoint (~134 MB) is stored with Git LFS. If you clone without it, you get a 134-byte placeholder file instead of the model — the clone appears to succeed, and MATS then can't detect markers.

# macOS:          brew install git-lfs
# Debian/Ubuntu:  sudo apt install git-lfs
# Conda:          conda install -c conda-forge git-lfs
# Windows:        included with Git for Windows
# RHEL/Fedora:    sudo dnf install git-lfs

git lfs install          # one-time setup, per machine

Step 2 — clone and install

Install into a virtual environment, so MATS has its own copy of Python:

git clone https://github.com/Breeding-Insight/Morphometric-Analysis-Toolbox-for-Segmentation.git
cd Morphometric-Analysis-Toolbox-for-Segmentation
python3.13 -m venv .venv                 # Windows: py -3.13 -m venv .venv
source .venv/bin/activate                # Windows: .venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -e ".[app]"        # ".[app]" adds the Streamlit GUI
mats --version                           # confirm the installed release version

Keep the -e: it lets MATS find the model in the folder you cloned. In each new terminal, activate the environment again (source .venv/bin/activate) before running mats.

The clone brings the RF-DETR checkpoint with it. Confirm it is the real file and not a placeholder — it should be ~134 MB, not ~134 bytes (ls -lh shows 128M, which is correct):

ls -l weights/rf_detr_marker.pth

Already cloned without Git LFS? No need to start over — install Git LFS as above, then repair the checkout in place. The --exclude keeps this to the ~134 MB RF-DETR file, avoiding the optional 2.65 GB BiRefNet checkpoint:

git lfs install && git lfs pull --exclude="weights/birefnet_leaf.pth"

Enhanced QR reading (optional). OpenCV reads QR codes well when they are oriented correctly and clearly visible. For images with any issues affecting the QR codes (glare, skew, blur) you can add the pyzbar + qreader fallbacks. pyzbar requires the system library zbar:

python -m pip install -e ".[app,qr]"

This enables QReader without Conda. To enable the additional pyzbar fallback, install its native zbar library with your existing environment:

# Linux: apt install libzbar0
# macOS: brew install zbar
# Existing Conda environment only: conda install -c conda-forge zbar

If a code can't be read, the pipeline continues — pass the finished sheet size with --sheet-dimensions (for example, --sheet-dimensions 12x12in), so enhanced QR is a convenience rather than a requirement.

Finally, confirm the environment:

mats doctor             # checks weights, GPU/CPU device, QR backends

Run mats doctor after installing, this will report MATS operable status. Note: RF-DETR weights are REQUIRED for marker detection, BiRefNet is OPTIONAL.

mats fetch-weights                               # repairs a clone made without Git LFS
mats fetch-weights --only birefnet --source lfs  # optional: the ~2.65 GB BiRefNet checkpoint

---

Model weights

The checkpoints for both the marker detection model and the leaf segmentation model are located in this repository, tracked with Git LFS:

Model	File	Size
RF-DETR marker detector	rf_detr_marker.pth	~134 MB
BiRefNet leaf segmenter	birefnet_leaf.pth	~2.65 GB

By default, only the RF-DETR model checkpoint will be downloaded: it arrives with every git clone made with Git LFS installed, so a normal checkout is immediately runnable. The BiRefNet checkpoint is LFS-tracked but excluded from the default clone, so it is downloaded only through an explicit action:

• Otsu (default) — needs no BiRefNet checkpoint and never downloads one.
• BiRefNet (optional) — fetch explicitly with mats fetch-weights --only birefnet --source lfs, or use the setup page.
• Shared filesystem — set MATS_WEIGHTS_DIR (e.g. a SCINet /project path) to read weights in place with no per-user copy.

Cloned without Git LFS? Both files come through as ~134-byte pointer stubs rather than models, which MATS detects and reports rather than handing to PyTorch. Fix it with git lfs install && git lfs pull --exclude="weights/birefnet_leaf.pth", or mats fetch-weights.

Full details and checksums: docs/weights.md.

Why the first installation is lightweight

MATS installs in a lightweight operating configuration for convenience. The standard app includes fast Otsu segmentation and OpenCV's built-in QR reader, but it does not automatically download the optional ~2.65 GB BiRefNet checkpoint or install the pyzbar/QReader robust-QR fallbacks.

The required ~134 MB RF-DETR marker checkpoint is different: it is mandatory for every run, so it ships in the clone via Git LFS and needs no separate download step. If it is missing from a Git checkout — for example, after cloning without Git LFS — MATS can fetch it once on first use and prints Fetching weights/rf_detr_marker.pth via Git LFS ... while it does. An install outside a Git checkout must use a pre-staged checkpoint (or a separately configured Hugging Face source). Set MATS_NO_AUTO_FETCH=1 to turn automatic fetching off and require pre-staged weights instead (recommended on HPC login nodes). The app never does this silently: a missing RF-DETR checkpoint is a blocking Preflight error.

This keeps the initial network and disk footprint predictable, avoids native zbar failures on managed machines, and works better on HPC systems and restricted networks. Start with the standard path, then add only what the photographs require:

Capability	Included initially	Add when needed
Otsu leaf segmentation	Yes	Nothing
Clear QR codes with OpenCV	Yes	Nothing
RF-DETR marker detection	Yes — checkpoint ships in the clone (Git LFS)	Nothing
BiRefNet segmentation	No checkpoint	mats fetch-weights --only birefnet --source lfs
Robust QR fallbacks	No	python -m pip install "mats-morpho[app,qr]"

QReader can download its detector model when that fallback is first used. pyzbar requires the native zbar library described above. The app's Preflight and setup pages show exactly which readers and checkpoints are available before a run.

---

Running MATS

• I want to click buttons → Using the app
• I want to script it → Using the command line
• I have a question → FAQ

Both run the exact same pipeline and produce the same measurements.

---

Using the app

MATS comes with a point-and-click user interface. To open it, simply run:

mats app

This will open the Streamlit app locally in your web browser. From there:

1. Pick images — a local folder, or drag-and-drop uploads.
2. Set the scale — enter the printed sheet's width, height, and unit (e.g. 10.5 x 9.5 in), or tick Variable dimensions, read QR code to read it from each image's template QR code automatically.
3. Choose segmentation — Otsu threshold (fast, default) or BiRefNet (accurate, must have local model checkpoint installed).
4. Choose workers — the app detects the number of CPUs available to it. One worker uses CUDA/MPS when available; two or more workers use parallel CPU processing and disable CUDA/MPS for that run. A colored warning light shows CPU allocation; counts above 75% require a one-run break-glass acknowledgement.
5. Run, then preview results and download a CSV or a ZIP of masks + boxes.

Getting oriented. The app's Help page (sidebar) ships three annotated sample photographs — including a real QR-read failure that shows why printed sheet entry is the most consistent option — a photography checklist, and a glossary for every results-CSV column. See docs/gui.md.

Printing templates. The app has a Template Creator page (in the sidebar) that accepts only the finished sheet's width and length, then automatically sizes the observation area and corner markers. Download the print-ready PDF or an editable Adobe InDesign IDML; the PDF is recommended for final printing. Print at 100% scale (no "fit to page"), lay your leaves inside the box, and photograph it flat. See docs/templates.md.

---

Using the command line

mats run -i ./images -o ./out -r results.csv --sheet-dimensions 12x12in

Common options (full reference in docs/cli.md):

Flag	Meaning	Default
-i, --input_dir	Folder of images to measure	prompt
-o, --output_dir	Where masks / target boxes are written	prompt
-r, --results_path	Measurement CSV path	./leaf_morpho_results.csv
--sheet-dimensions	Finished Template Creator sheet size, <w>x<h><unit>	read from QR
-t, --template_dimensions	Legacy/custom marker-centre calibration area	unused
--mask-method	birefnet (accurate, GPU), threshold (fast), or both	threshold
--threshold-level	auto (Otsu) / low / medium / high, or a custom cutoff 1–255	auto
--csv-schema	full (area/width/length + per-axis pixels-per-selected-unit) or compact	full
--results-unit	Measurement-output unit: mm, cm, or in	cm
--measure-pre-cleanup	Measure from raw binary masks before cleanup, minus the edge margin and stray pieces	off
--clean-margin	With --measure-pre-cleanup: band cleared along the target-box edge, as a percent of its shorter side	1
--stray-gap	With --measure-pre-cleanup: how far a piece may lie from the leaf, as a fraction of its bounding-box diagonal	0.25
--clean-size	With --measure-pre-cleanup: remove specks and fill enclosed holes smaller than this radius in pixels (the app's Clean size)	0 (off)
-w, --workers	Parallel workers (threshold path only)	auto
--save-axes	Also save length/width overlay images for QC	off

Choosing a segmentation method. threshold (Otsu) is the default — fast, no GPU, no extra download, and good for clean, high-contrast backgrounds where a leaf sits on plain white. birefnet is more accurate on cluttered or low-contrast backgrounds and uses a GPU when available (CPU works but is slow), at the cost of the ~2.65 GB checkpoint — fetch it once with mats fetch-weights --only birefnet. To compare them, --mask-method both (or checking both methods in the app) measures every image with each method and writes one results CSV per method.

---

Outputs

Per image, in the output folder:

• {sample_id}_target_box.jpg — the perspective-corrected observation box
• {sample_id}_mask.png — the leaf segmentation mask

Both image exports are on by default and can be disabled with --no-target-boxes or --no-masks. Pre-cropped target-box inputs are not copied. Optional --export pre-cleanup saves each measurement method's binary mask before cleanup; --pre-cleanup-methods both runs Otsu/threshold and BiRefNet and saves their pre-cleanup masks separately. Only a --mask-method method determines measurements. BiRefNet exports require its checkpoint to be installed locally. With --mask-method both, each method's masks and QC images end in _threshold or _birefnet (for example {sample_id}_mask_birefnet.png). --export overlay, --export cutout, and --export axes add QC images; repeat the flag to request multiple kinds. The app offers the same choices in Setup. Use --measure-pre-cleanup (or Measure from pre-cleanup masks in the app) to calculate measurements from the raw binary segmentation. A thin band along the target-box edge, where the template's printed box outline lands, is cleared first (--clean-margin). The largest remaining object is taken as the leaf; other pieces that touch that band, or lie farther from the leaf than --stray-gap times its bounding-box diagonal, are dropped. Area counts the remaining foreground pixels, and width and length span their extent, so specks near the leaf can still affect the result; --clean-size (for example --clean-size 3) removes specks and fills enclosed holes whose inscribed radius is below that many pixels, without flash-filling the leaf. This choice is independent of --export pre-cleanup, which saves the mask with every piece. Each results CSV has a .meta.json companion recording the MATS version, input filenames, calibration input and successful per-image calibration, method, and effective Otsu mode/cutoff where used. Pre-cleanup runs also record their cleanup settings; saved app adjustments are recorded per specimen.

Plus a measurements CSV. Choose mm, cm (the default), or in with --results-unit in the CLI or the Result units control in the app. The selection changes results, dashboard labels, and unit-bearing CSV column names; it does not change calibration math. Two schemas:

• full (default, research schema) — sample_id, leaf_area_cm2, width_cm, length_cm, px_per_cm_width, px_per_cm_height, scale_aspect_ratio, source. When dimensions are read from QR codes, it also appends a trace column for OpenCV and each optional decoder installed locally, showing which decoder succeeded or whether a fallback failed or was unused. Scaling is anisotropic: the x-extent (width_cm) is divided by px_per_cm_width, the y-extent (length_cm) by px_per_cm_height, and area by their product — each axis calibrated independently against the template, rather than one averaged scalar applied to everything. scale_aspect_ratio (px_per_cm_width / px_per_cm_height) is a QC signal: it should sit near 1.0, and a value far from 1.0 flags a calibration problem (skewed template print, lens distortion, a non-planar sheet) worth investigating.
• compact — sample_id, area_cm2, width_cm, length_cm by default. With millimeters or inches selected, cm is replaced consistently in the measurement column names.

A leaf_morpho_failures.csv records per-image warnings and failures. With --mask-method both (or both methods checked in the app), each method writes its own results CSV and failure log with a method suffix, for example leaf_morpho_results_birefnet.csv and leaf_morpho_failures_birefnet.csv, in the same schema as a single-method run.

Migration note: earlier versions reported three isotropic scale conventions (*_meanscale, *_widthscale, *_heightscale). Old CSVs remain usable — the new leaf_area_cm2 can be recovered from an old row with leaf_area_cm2_widthscale * (px_per_cm_width / px_per_cm_height), and the new width_cm/length_cm equal the old width_cm_widthscale/length_cm_heightscale.

---

Running on a compute cluster

An Open OnDemand Batch Connect app that serves the GUI on a compute node is in deploy/ondemand/mats/. See its README and docs/hpc.md.

---

How it works

MATS chains two models. RF-DETR (fine-tuned, single "Marker" class) detects the four corner fiducials at 1120×1120 px; their centroids define a homography that rectifies the observation box. The rectified box's pixel width and height are compared against the template's known physical size, independently per axis, to fix px_per_cm_width and px_per_cm_height. BiRefNet (fine-tuned for leaf foreground) then segments the leaf, from which area (pixel count) and length/width (bounding dimensions) are computed and converted to centimeters using their respective axis scale. A classic Otsu threshold is offered as a fast alternative to BiRefNet. See the manuscript for training and evaluation detail.

---

Troubleshooting

Run mats doctor first — it reports most of these, and the FAQ covers the common questions in more detail.

• The install fails, or mats / pip is "command not found" — look up the exact message in the table in docs/install.md. Most install failures come from a Python older than 3.10 (including the one built into macOS) or from installing outside a virtual environment.
• QR code not read / measurements need a scale — the default OpenCV decoder couldn't read the code. Pass the finished sheet size with --sheet-dimensions (e.g. --sheet-dimensions 12x12in), or add enhanced QR reading: python -m pip install -e ".[qr]" plus the zbar system lib (Linux: apt install libzbar0; macOS: brew install zbar; conda: conda install -c conda-forge zbar).
• CUDA out of memory (only relevant with --mask-method birefnet) — process in smaller batches, or use --mask-method threshold (the default).
• No markers detected / "RF-DETR checkpoint missing" — first check that the checkpoint is a real file and not a Git LFS placeholder: ls -l weights/rf_detr_marker.pth should show ~134 MB, not ~134 bytes. If it's a placeholder, run git lfs install && git lfs pull --exclude="weights/birefnet_leaf.pth". Otherwise check print quality and that the marker color matches the template (the Template Creator uses the trained color); make sure all four corners are in frame.
• Blank page on Open OnDemand — almost always the reverse-proxy baseUrlPath mismatch; see deploy/ondemand/mats/README.md.

---

Working with an AI assistant

This repository ships agent instructions in AGENTS.md (with companion CLAUDE.md and GEMINI.md files for the tools that don't read AGENTS.md on their own), so a coding assistant you point at your clone — Claude Code, Codex, Cursor, Copilot, Gemini CLI — already knows how MATS is installed, run, and structured, and can help you troubleshoot a batch.

Reproducibility and citing

For a published analysis, retain the MATS version (mats --version), the exact Git tag or commit, and the version-specific software DOI when available. Record the checkpoint filenames and SHA-256 hashes from docs/weights.md, the segmentation method and threshold setting, the printed sheet dimensions or QR-derived calibration, and the source-image identifiers. Keep the results CSV, any failure CSV, and its .meta.json companion together. If you adjust an Otsu specimen in the app, retain the updated metadata and masks with the revised CSV.

Please cite the MATS software release using CITATION.cff and the companion manuscript when its citation is available. A version-specific software DOI identifies the archived code; a separate weights DOI identifies the model files. The Python distribution is mats-morpho, and its command is mats.

License

MIT. The pipeline builds on RF-DETR (Apache-2.0) and BiRefNet (MIT); see docs/weights.md for model provenance.