DeepPhD: Physics-informed self-supervised denoising for ultrasensitive fluorescence microscopy

Project page | Paper

Contents

• Overview
• Repository structure
• Installation
• Quick start
• Results

Overview

Fluorescence microscopy is fundamentally limited by noise, which compromises imaging sensitivity and obscures biological phenomena. Noise originating from different optoelectronic sources exhibits distinct statistical properties. The heterogeneity of noise poses critical challenges for reliable noise removal.

DeepPhD (Deep Physics-informed Denoising) is a physics-informed, self-supervised denoising framework that synergizes image restoration with noise physics. By explicitly modeling heterogeneous noise components within a learnable flow and informing the image restoration module of noise parameters, DeepPhD reinforces noise decoupling and signal estimation without requiring any clean images, thereby resolving fluorescence signals from severe noise and improving downstream quantitative analyses.

We demonstrate the superiority of DeepPhD on various imaging modalities and biological processes, including light-sheet imaging of GABAergic neurons in larval zebrafish, widefield neural imaging of freely behaving mice, and multiphoton imaging of immune cell migration. DeepPhD extends the performance and interpretability of fluorescence image denoising and facilitates reliable biological observation under photon-limited conditions.

Repository structure

👆Click to unfold the directory tree

DeepPhD/
├── DeepPhD_train.py          # Training entry point
├── DeepPhD_inference.py      # Inference entry point
├── requirements.txt          # Pinned dependencies (excluding PyTorch)
├── model/
│   ├── DeepPhD.py            # Joint physics model and 3D U-Net
│   ├── network/              # 3D U-Net denoiser
│   └── noise_model/          # FPN, RN, and MPGN modules
├── data_loader/              # Patch extraction, augmentation, and dataloaders
└── utils/
    ├── arg_parser.py         # CLI parsing, GPU setup, and checkpoint utilities
    └── inference_io.py       # Patch-wise inference and TIFF I/O

Installation

🖥️ System requirement

• Linux (recommended)
• Python 3.10
• NVIDIA GPU with CUDA 12.x
• A recent PyTorch build compatible with your GPU (select the matching CUDA wheel on pytorch.org)

⚙️ Environment configuration

git clone https://github.com/cabooster/DeepPhD.git
cd DeepPhD
conda create -n deepphd python=3.10 -y
conda activate deepphd

Install a PyTorch version that is compatible with your CUDA version and GPU. Use the selector on pytorch.org to choose a version compatible with your driver and hardware. Newer GPUs, such as the RTX 5090, need a recent PyTorch version to work properly. Example for CUDA 12.8:

pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 \
    --index-url https://download.pytorch.org/whl/cu128

Install the other dependencies:

pip install -r requirements.txt

📁 Data format

Organize input image stacks as multi-page TIFF stacks (.tif) in a single folder, for example:

your_dataset/
  ├── stack_001.tif
  └── stack_002.tif

After being loaded by the Python code, each TIFF should have a shape of T × H × W (frames × height × width). Stacks with fewer than 400 frames are automatically extended by reflection padding to meet the minimum length required for training.

All stacks in the same data folder must come from the same imaging device (sensor), so that the noise parameters stay consistent within a single training or inference run.

Quick start

1. Noise model

Considering the noise sources in fluorescence imaging, the overall noise model can be formulated as the additive combination of mixed Poisson–Gaussian noise (MPGN), fixed-pattern noise (FPN), and row noise (RN):

Noise component	Description
MPGN	Photon shot noise (Poisson noise), dark noise (Gaussian noise), readout noise(Gaussian noise).
FPN	Pixel-wise nonuniformity, modeled as a time-invariant 2D pattern.
RN	Row-wise nonuniformity, modeled as time-varying stripe noise, with all pixels in each row sharing the same value.

Choose the noise configuration that best matches your data. The table below lists our recommendations:

Detector	Typical modalities	Recommended --noise_model
CMOS/sCMOS camera	Light-sheet microscopy, widefield microscopy, light-field microscopy, etc.	fpn|rn|mpgn
CCD/EMCCD camera	Singlemolecule localization microscopy (SMLM), etc.	fpn|mpgn
Photomultiplier tube (PMT)	Two-photon microscopy, three-photon microscopy, etc.	mpgn

2. Training

python DeepPhD_train.py \
  --exp_dir demo_lightsheet_zebrafish \
  --datasets_path /path/to/your_dataset \
  --noise_model fpn|rn|mpgn \ 
  --save_noise

By default, training runs on GPUs 0 and 1. To use different GPUs, specify them with --gpu (e.g., --gpu 0 or --gpu 0,1,2). Other key arguments are listed below:

Argument	Description
--exp_dir	Output root directory. Logs and checkpoints are saved under results/<exp_dir>/.
--datasets_path	Directory containing input .tif stacks.
--noise_model	Noise configuration that best matches your data. e.g., fpn|rn|mpgn, fpn|mpgn, or mpgn (default: fpn|rn|mpgn).
--gpu	Comma-separated GPU IDs (default: 0,1).
--fresh_start	Restart training from scratch.
--save_noise	Save the learned FPN and RN patterns.
--seed	Random seed (default: 0).

Models (training checkpoints) are saved as:

results/<exp_dir>/saved_models/epoch_<N>.pth

Denoising results (and optional noise maps) are saved under results/<exp_dir>/.

3. Inference

python DeepPhD_inference.py \
  --exp_dir demo_lightsheet_zebrafish \
  --datasets_path /path/to/your_dataset \
  --noise_model fpn|rn|mpgn \
  --save_noise

Argument	Description
--exp_dir	Output root directory.
--epoch	Determine which checkpoint to load (default: the latest epoch).
--noise_model	Noise configuration for inference. Must match the noise configuration used for training.
--datasets_path	Directory of TIFF stacks to denoise.
--gpu	Comma-separated GPU IDs (default: 0,1).
--save_noise	Export estimated RN and learned FPN maps.

Results

1. Ultrasensitive light-sheet imaging of GABAergic neurons in larval zebrafish with DeepPhD.

2. High-fidelity neural recordings from freely behaving mice with head-mounted miniaturized microscopy.

3. Calcium transients in dendritic spines revealed in the mouse cortex.