Problem#
ECG machines are everywhere, inside and outside the clinic, so passive arrhythmia monitoring is possible. But labelled ECG data is scarce, imbalanced, and noisy, and supervised models overfit to those problems.
Approach#
- Masked Patch Modelling (MPM): a self-supervised pre-training objective that masks patches of the ECG signal and learns to reconstruct them.
- Large-scale pre-training on 8.2 million unlabelled ECGs to produce PatchECG, a foundational 1D transformer that can be fine-tuned for any downstream ECG task.
- Ablations on other remedies for class imbalance, label noise, and over-parameterization.
Results#
- State-of-the-art on standard benchmarks, including PTB-XL multi-label classification, and new benchmarks on the largest, highest-quality multi-label dataset to date.
- Outperforms the previous state-of-the-art using 1/5 of the compute while increasing model capacity 14×.
- Significantly outperforms a state-of-the-art 2D vision transformer.
- More than 2% improvement under class imbalance, label noise, and over-parameterization.
