Errors that matter: Uncertainty-aware universal machine-learning potentials calibrated on experiments
Matthias Kellner, Teitur Hansen, Thomas Bligaard, Karsten Wedel Jacobsen, Michele Ceriotti
Abstract
Machine-learning models of atomic-scale interactions achieve the accuracy of the quantum mechanical calculations on which they are trained, but at a dramatically lower computational cost. Their predictions can be made trustworthy by uncertainty quantification techniques that estimate the residual error relative to their reference. These errors, however, do not include uncertainty contributions from the approximations inherent in the electronic structure calculations, which are often the main source of discrepancy with empirical observations. We construct an ensemble of ML potentials trained on multiple electronic-structure references and calibrate it against experimental data on cohesive energies, atomization energies, lattice constants and bulk moduli of simple materials and molecules, similar to the uncertainty-aware functional distribution approach. The resulting ensemble of models, which we call PET-UAFD, can be used to simulate matter across a wide range of compositions and thermodynamic conditions. By comparison with experimental measurements of the density and structure of liquids, we demonstrate that, even outside the static properties on which it was calibrated, PET-UAFD enables predictions that are as accurate against experiments as the best available electronic-structure reference and that the spread in the ensemble can be used to assess the reliability of such predictions. We also introduce the PET-EXP protocol that uses shallow ensembles and statistical reweighting techniques to provide accurate estimates of uncertainty relative to experimental measurements at virtually no additional cost over a simulation based on a single conventional ML potential. Ultimately, this approach provides a practical and inexpensive approach to elevate machine-learning potentials from faithful interpolators of approximate theories to genuinely predictive tools anchored in experimental reality.
AI Impact Assessments
(3 models)Scientific Impact Assessment: "Errors that matter: Uncertainty-aware universal machine-learning potentials calibrated on experiments"
1. Core Contribution
This paper addresses a fundamental gap in the machine-learning interatomic potential (MLIP) ecosystem: while MLIPs faithfully reproduce their DFT training references and uncertainty quantification (UQ) methods can estimate the ML-to-DFT residual error, neither captures the more consequential error — the discrepancy between DFT and physical reality. The authors construct PET-UAFD, an ensemble of universal MLIPs trained on five different exchange-correlation functionals (PBE, PBEsol, PBE+U, r2SCAN, r2SCAN-D3), calibrated against experimental measurements of cohesive energies, atomization energies, lattice constants, and bulk moduli. The key innovation is twofold: (1) extending the UAFD framework from direct DFT calculations to MLIP surrogates, enabling evaluation of complex thermodynamic properties that would be prohibitively expensive with DFT; and (2) introducing PET-EXP, which combines shallow multi-head ensembles with statistical reweighting (DPOSE) to deliver experiment-calibrated UQ at virtually zero computational overhead beyond a single MLIP simulation.
2. Methodological Rigor
The methodology is well-grounded, building on the established UAFD framework of Hansen et al. (2025) and the DPOSE scheme of Kellner and Ceriotti (2024). The mathematical formulation is clean: a Gaussian distribution over weights in the MLIP model space is optimized by maximizing likelihood against experimental data (Eq. 1), with the minimal ensemble constructed via eigendecomposition of the covariance matrix (Eq. 3). The variance estimation (Eq. 2) is exact to second order in parameter variations, which is appropriate given the near-linear dependence of many observables on the mixing weights.
The experimental validation is thorough — 5-fold cross-validation on training properties shows well-calibrated uncertainties (RMSNE ≈ 1) for atomization energies, lattice constants, and bulk moduli, with cohesive energies showing modest underestimation by ~2×. The truly compelling validation is the extrapolative test on 21 liquid systems, evaluating densities and radial distribution functions (RDFs) that were not part of calibration. The correlation between predicted uncertainties and actual errors spans several orders of magnitude (Fig. 2b), providing strong evidence that the uncertainty estimates are meaningful outside the training domain.
One methodological concern is the small basis of five functionals, all from the GGA/meta-GGA rung — this limits the diversity of the model space. The authors acknowledge this and note that DPOSE makes scaling to larger bases feasible. The outlier removal process (Section S4) also requires careful scrutiny; removing data points that don't fit well could bias the calibration, though the justifications provided (cutoff limitations, incorrect spin states) seem physically reasonable.
3. Potential Impact
This work has high practical impact for the computational materials science community. Universal MLIPs are rapidly becoming standard tools, and anchoring their predictions to experimental reality rather than to a single DFT functional of unknown fidelity addresses a genuine need. The PET-EXP protocol is particularly impactful: it transforms any MLIP simulation into an experiment-calibrated uncertainty estimate at negligible additional cost, lowering adoption barriers dramatically.
The demonstration on liquid-state properties is strategically chosen — these are precisely the kind of finite-temperature, large-system-size problems where MLIPs provide maximum advantage over DFT, yet where DFT functional errors can accumulate unpredictably. The finding that w₀ matches the accuracy of the best individual functional (r2SCAN-D3) while providing calibrated error bars is practically valuable.
The "invariant point" analysis (Section IV.C) is a notable scientific insight: the discovery of isosbestic-like crossings in g(r) from different functionals, explained by varying populations of two local environment types, provides mechanistic understanding of how DFT functional choice affects liquid structure predictions.
4. Timeliness & Relevance
This paper is exceptionally timely. The field is experiencing rapid proliferation of universal MLIPs (MACE-MP-0, CHGNet, UMA, PET-MAD), yet the community lacks standardized approaches for assessing when these models' predictions can be trusted for practical decisions. The "accuracy ceiling" imposed by DFT approximations is widely acknowledged but rarely quantified for complex properties. This work provides a concrete, implementable solution.
The approach also arrives at a moment when multiple large-scale DFT databases (MATPES, MAD, OMAT24) with different functionals are becoming available, making multi-functional MLIP training feasible without new DFT campaigns.
5. Strengths & Limitations
Key Strengths:
Notable Limitations:
Additional Observations
The paper elegantly reframes the purpose of MLIPs — from interpolators of a single approximate theory to calibrated predictive tools. The intellectual contribution of combining UAFD with DPOSE is greater than the sum of parts, as it resolves a practical impossibility (running UAFD with DFT for liquids) through an elegant computational framework. The scalability to more diverse functional bases, potentially including beyond-DFT methods, represents a clear and exciting future direction.
Generated Apr 28, 2026
Comparison History (61)
Paper 2 is likely to have higher scientific impact because it tackles a central, widely recognized limitation of ML interatomic potentials: mismatch between DFT-trained models and experimental reality. By combining multi-reference training with explicit calibration to experiments and providing practical uncertainty estimates (PET-UAFD, PET-EXP) at near-single-model cost, it offers broadly applicable methodology that can change how potentials are validated and used across materials chemistry and condensed-matter simulation. Paper 1 is a strong, rigorous software/engineering advance, but its impact is more incremental and tool-focused within an existing HDNNP ecosystem.
Paper 1 likely has higher impact: it introduces a concrete, experimentally calibrated, uncertainty-aware ML potential framework (PET-UAFD) plus a practical low-cost protocol (PET-EXP) that can immediately improve predictive materials/molecular simulations across compositions and conditions. This is methodologically innovative (integrating multi-reference electronic-structure uncertainty with experimental anchoring) and highly timely given the push for trustworthy ML interatomic potentials. Paper 2 is a valuable, broad review of stochastic resetting with cross-domain relevance, but as a review it is less novel and its impact is more diffuse and dependent on downstream adoption.
Paper 2 likely has higher scientific impact due to its broadly applicable framework: uncertainty-aware “universal” ML potentials that explicitly account for electronic-structure model discrepancy and are calibrated to experiments. This tackles a central barrier to real-world adoption—trustworthy, experimentally anchored predictions with quantified reliability—across many materials, phases, and conditions. Methodologically, the multi-reference ensemble + experimental calibration + low-cost PET-EXP uncertainty protocol is general and timely. Paper 1 is highly innovative and impactful for biomolecular implicit-solvent simulation, but its domain scope is narrower than Paper 2’s cross-field applicability.
Paper 1 addresses a fundamental limitation of ML potentials—their disconnection from experimental reality—by calibrating against experimental data and providing meaningful uncertainty quantification across diverse chemical systems. This universally applicable framework (PET-UAFD/PET-EXP) bridges theory and experiment for general atomistic simulations, impacting computational chemistry, materials science, and condensed matter broadly. Paper 2, while impressive in scaling generative models to explicit solvent systems for hydration free energies, addresses a narrower application domain. Paper 1's methodological innovation of anchoring ML potentials to experiments has broader transformative potential across scientific disciplines.
Paper 1 addresses a fundamental limitation in atomistic machine learning by calibrating ML potentials directly against experimental data rather than relying solely on approximate quantum mechanical calculations. Providing reliable uncertainty quantification relative to experimental reality has profound implications across materials science and chemistry, elevating ML potentials from theory interpolators to genuinely predictive empirical tools. While Paper 2 offers an innovative compositional approach for generative modeling of large systems, Paper 1's foundational improvement to the accuracy and reliability of molecular dynamics simulations promises a broader and more transformative impact on computational sciences.
Paper 1 presents a highly innovative approach by calibrating machine-learning potentials against experimental data rather than just approximate electronic-structure calculations. This directly addresses the fundamental limitation of training on imperfect theoretical references. By providing computationally inexpensive, uncertainty-aware predictions anchored in experimental reality across diverse thermodynamic conditions, it offers a universally applicable tool with massive potential impact across materials science, physics, and chemistry. Paper 2 is strong methodologically but focuses more narrowly on electronic Hamiltonian prediction, whereas Paper 1 shifts the broader paradigm of molecular dynamics simulations.
Paper 2 addresses a fundamental limitation of ML potentials—their reliance on approximate electronic structure calculations—by calibrating models directly against experimental data and providing uncertainty quantification. This paradigm shift from interpolating approximate theories to predicting experimental reality offers broader conceptual impact and greater reliability across materials science and chemistry, whereas Paper 1 focuses primarily on computational speed-ups via model distillation.
Paper 1 addresses a fundamental limitation of ML potentials—their calibration against approximate electronic structure methods rather than experimental reality. By constructing uncertainty-aware ensembles calibrated on experiments (PET-UAFD) and introducing the cost-effective PET-EXP protocol, it transforms MLIPs from interpolators of theory into genuinely predictive tools. This paradigm shift has broad impact across computational chemistry, materials science, and molecular simulation. Paper 2 offers a valuable but more incremental contribution on sample efficiency through multitask learning with semiempirical charges, which is important but narrower in scope and conceptual novelty.
Paper 2 likely has higher impact due to its broadly applicable, timely methodology: uncertainty-aware universal ML interatomic potentials calibrated to experiments, addressing a major barrier (DFT-to-experiment discrepancy) in computational materials/chemistry. It offers practical protocols (PET-UAFD, PET-EXP) that can be adopted across many systems and fields (materials science, chemistry, geoscience, catalysis), improving predictive reliability with quantified uncertainty at low cost. Paper 1 is novel and valuable for nanoconfined ion transport and energy applications, but its impact is narrower and more system-specific.
Paper 1 likely has higher impact due to its direct linkage of ML interatomic potentials to experimental calibration plus uncertainty quantification, addressing a central barrier to real-world deployment of ML potentials across chemistry/materials. It is timely and broadly applicable (simulations across compositions/conditions, liquids, materials discovery), and the PET-EXP low-cost protocol could see wide adoption. Paper 2 is methodologically rigorous and valuable for scaling correlated electronic structure (2RDM compression, eigenvector continuation), but its applications are more specialized to high-level quantum chemistry workflows, likely narrowing breadth and near-term uptake compared to experimentally anchored universal ML potentials.