FedCompress

Communication-Efficient Federated Learning through
Adaptive Weight Clustering and Server-Side Distillation

Vasileios Tsouvalas · Aaqib Saeed · Tanir Özçelebi · Nirvana Meratnia

Abstract

FedCompress reduces communication in Federated Learning (FL) through adaptive weight clustering and server-side distillation. Clients train with shared weight centroids to compress their uploads. After aggregation, a clustered student learns from the global model using unlabeled auxiliary data, restoring compression for the return trip. A client-computed representation-quality score adapts the number of clusters as learning progresses. Across five vision and audio datasets, the paper reports 3.80–5.41× lower communication, with 0.31–1.88 percentage-point accuracy reductions relative to FedAvg.

Setup

Create and activate the fedcompress environment:

git clone https://github.com/Nousphera/FedCompress.git
cd FedCompress
python3.12 -m venv --prompt fedcompress "$HOME/.venvs/fedcompress"
source "$HOME/.venvs/fedcompress/bin/activate"

From the activated fedcompress environment, install the requirements:

python -m pip install -r requirements.txt

Data and Weights

Download, prepare and validate the data:

# CIFAR-10 and auxiliary images for server distillation
python scripts/prepare_data.py --dataset cifar10
python scripts/prepare_data.py --dataset stylegan

# All datasets and auxiliary corpora
python scripts/prepare_data.py

Datasets: cifar10 · cifar100 · pathmnist · spcm · voxforge.
Auxiliary data: stylegan for vision · librispeech for audio.

For VoxForge, download the listed archives and retain their language folders under raw/voxforge/, for example the de/Trunk/Audio/Main/16kHz_16bit/ subfolder. Preparation checks the complete archive list and creates speaker-disjoint splits.

Prepare Model

python scripts/prepare_model.py

Note

ResNet-20 handles images; MobileNet/YAMNet handles audio. Both train from scratch. The script saves their seeded initializations; use the same --seed for preparation and experiments.

Run Experiments

From the repository root, launch your chosen method as a Flower App:

# Shared experiment settings
common=(--dataset cifar10 --nr-clients 20 --rounds 20
        --participation 1.0 --quantity-skew 0.25
        --local-epochs 10 --warmup-epochs 5 --server-epochs 10
        --batch-size 128 --min-clusters 8 --max-clusters 30 --device cuda)

# FedCompress (ours)
python scripts/run_experiment.py "${common[@]}" --method fedcompress

# Client-side clustering · without server self-compression
python scripts/run_experiment.py "${common[@]}" --method client

# FedAvg
python scripts/run_experiment.py "${common[@]}" --method fedavg

Note

Clustering follows five warm-up epochs. FedCompress then distills a clustered server model on auxiliary data; the baselines need no auxiliary corpus. All methods use the same local training budget and sample-weighted aggregation.

Citation

@inproceedings{tsouvalas2024fedcompress,
  title = {Communication-Efficient Federated Learning through Adaptive Weight Clustering and Server-Side Distillation},
  author = {Vasileios Tsouvalas and Aaqib Saeed and Tanir Ozcelebi and Nirvana Meratnia},
  booktitle = {ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  year = {2024},
  pages = {5805--5809},
  doi = {10.1109/ICASSP48485.2024.10447174}
}

Component Licenses