Leakage-Controlled Benchmarking of Variant Effect Predictors and Development of Gradient-Boosting Consensus Model

Gradient-boosting consensus meta-predictor, evaluated on a benchmark dataset controlled for data circularity

The authors reviewed 18 recent and 3 older, widely adopted variant effect predictors and assembled a benchmark dataset controlled for data circularity, combining clinically annotated human variants (ClinVar, Humsavar) with functionally annotated cross-species variants. Tool performance fell within a narrow range (AUC 0.631–0.758), with a consistent drop from clinical to functional data. Several tools published two decades ago performed comparably to mid-ranked modern predictors. The gradient-boosting consensus model PredictSNPprot reached an AUC of 0.82 on a held-out test set, outperforming the original PredictSNP meta-predictor by 13.6 and the best individual tool by 5.7 percentage points. The benchmark dataset (Zenodo) and the code (GitHub) are publicly available.

Synthesis written by Geno'X. For the full original abstract, please refer to the source publication.

Preprint, not peer reviewed. The main message is the benchmark: once circularity is controlled, missense predictors are closely bunched, which argues against over-interpreting the gap between two scores when applying PP3 or BP4. An AUC of 0.82 remains modest for diagnostic use, and the abstract does not report the consensus performance on the human clinical subset alone.

Analysis by Dr Thibaut Benquey