RVCBench (accepted at NeurIPS 2026) combines conventional voice-cloning utility evaluation with tests across speakers, languages and real-world audio conditions.
- Utility: speaker similarity, content accuracy, speech quality, intelligibility and emotion consistency.
- Robustness: noisy or overlapping references, compression, anti-cloning protection, long-form speech, unusual text, speaker demographics and multilingual/cross-lingual coverage.
- Automated evaluation: a pip package with seven metrics, prompt export and evaluation reports. Use our datasets or score your own audio.
- Cloned speech for deepfake research: 51,562 voice-cloned clips from 19 open-source systems, each with its prompt and a real recording of the same text.
Linux, Python 3.10–3.13 and FFmpeg are required. CPU and GPU scoring are supported.
python -m pip install "rvcbench[eval]"
rvcbench setup-scorerssetup-scorers downloads and verifies metric models once. For FFmpeg setup, CPU/GPU installation
and download requirements, see the installation guide.
| Your goal | Start here |
|---|---|
| Score generated audio with your own reference data | Score your own audio |
| Test a model on RVCBench datasets | Evaluate your model |
| Reproduce the published experiments | Reproduce the paper |
| Get cloned speech for deepfake detection or source tracing | Cloned speech dataset |
from rvcbench import metrics
with metrics.Evaluator(["sim", "wer", "speechmos"], device="cpu") as evaluator:
scores = evaluator.score(
"generated.wav",
reference="speaker_reference.wav",
text="Hello there.",
language="en",
)
print(scores)Replace the audio paths and expected text with your own. Available metrics are sim, sva, wer,
speechmos, mcd, stoi and emotion. MCD/STOI also require a same-text target recording.
See the metrics guide for inputs and batch scoring.
Start with the 52-utterance onboarding suite:
rvcbench prompts --suite onboarding-v1 --output prompts/
# Generate each prompt with your model; save outputs as <id>.wav in outputs/my-model/.
rvcbench score --suite onboarding-v1 --generated outputs/my-model --output results/my-model --device cpuRVCBench prepares the data and scores your outputs; you supply model inference. Prompt exports include JSONL, ZipVoice TSV and Seed-TTS-style batch lists. The step-by-step guide covers file formats, generation and reading the report.
For broader evaluation, use core-v1 (480 utterances) or full-v1 (12,724 utterances). See suite coverage and task selection and resume and model comparison. Packaged suites are currently previews and use a separate protocol from the paper. The paper's deepfake-detectability evaluation and audio-LLM expression judge are not included in these suites.
RVCBench Cloned Speech Samples provides 51,562 synthetic clips from 19 open-source voice cloning systems for audio deepfake detection, source tracing and anti-spoofing research. Each clip comes with the prompt the system was given and a real recording of the same text by the same speaker. The 12 conditions include clean, cross-lingual, long-form, noisy, multi-speaker and anti-cloning-protected prompts in English, Mandarin and French.
from datasets import load_dataset
ds = load_dataset("Nanboy/RVCBench-Cloned-Speech", split="train", streaming=True)
row = next(iter(ds)) # row["audio"], row["prompt"] and row["ground_truth"]The clips are licensed CC BY-NC 4.0, with additional terms for some systems listed on the dataset card.
The paper defines 18 evaluations, 14,370 utterances and 204 speakers. Browse interactive results and audio examples, paper result tables, or the paper for the full analysis.
The frozen v1 branch (tag v1.0)
contains the paper-release code. main contains the installable package for evaluating new models.
See versions and reproduction.
- Built-in models — 32 voice-cloning adapters, protection and denoising workflows.
- Datasets — source corpora, Hub folders and preprocessing.
- Python API · CLI reference · Run records.
- Contributing — development setup, checks and adding a model.
@inproceedings{jin2026rvcbench,
title = {RVCBench: Benchmarking the Robustness of Voice Cloning Across Modern Audio Generation Models},
author = {Jin, Ruinan and Liao, Xinting and Yu, Hanlin and Pandya, Deval and Li, Xiaoxiao},
booktitle = {Advances in Neural Information Processing Systems},
url = {https://arxiv.org/abs/2602.00443},
year = {2026}
}CC0-1.0. Models, upstream code and source corpora retain their own licenses. The cloned speech dataset is licensed separately under CC BY-NC 4.0. Questions and contributions are welcome through issues or ruinanjin@alumni.ubc.ca.