Radial Kernel Attention for Long-Range Modeling in Polymer Machine Learning Force Fields
Wenbo Fu, Chenghui Mao, Chengwen Hu, Haifeng Lv, Zhou Lv
Simple, clean global-attention add-on with strong OPoly26 numbers versus a much larger UMA, but single-seed, single-dataset, no mechanism ablations, and a centroid-based design with acknowledged physical limitations.
Abstract
We introduce Radial Kernel Attention (RKA) for long-range modeling in polymer machine learning force fields. Local neighbor graphs constrain communication across polymer chains and their surrounding environments, while explicit all-pair attention incurs quadratic cost. RKA maps atomic distances from the cluster centroid to learned radial features and combines their overlap with content-dependent queries and keys. Factorized aggregation and readout enable global feature exchange at linear cost in atom count for fixed feature dimensions. We insert the invariant scalar update between EquiformerV3 local blocks and obtain conservative forces by differentiating the total energy, including its radial coordinate dependence. We evaluate energy and force prediction for finite polymer-derived clusters in OPoly26. Our models are trained on the official training split for 12 epochs and evaluated directly after training on the complete official validation split. A matched 5M comparison shows that RKA reduces per-atom energy and force MAEs by 67.4% and 36.3%, respectively. The 20M RKA model is compared with UMA-S-1.2 and UMA-S-1.2.1 on the same validation structures. The 20M model attains a total-energy MAE of 15.81 meV and a force MAE of 2.619 meV/angstrom; its per-atom energy MAE is 0.1182 meV/atom. RKA provides a compact global communication pathway that improves energy and force prediction on the evaluated polymer-derived clusters.
AI Impact Assessments
(1 model)Core Contribution
The paper proposes Radial Kernel Attention (RKA), a global scalar-channel update for equivariant MLFFs. Each atom's distance from the cluster centroid is mapped to M=16 learned, normalized Gaussian radial features. The overlap of these features (K_rad = Σ_a φ_ia φ_ja) modulates a positive-feature linear attention between queries and keys. Factorization gives one key–value matrix per radial feature, so the cost is linear in atom count with no N×N matrix. The output is a zero-initialized, gated residual inserted into the ℓ=0 channels between EquiformerV3 local blocks. Forces come from differentiating total energy, with no stop-gradient on the centroid or radial map. The setting is OPoly26, a large DFT dataset of polymer clusters (≤360 atoms, median spatial extent 23 Å against a 6 Å cutoff).
The idea is a compact, invariant global mixer that is a hybrid of linear attention (Katharopoulos) and a radial basis kernel. It is neat but incremental. A centroid-relative radial kernel is a weak geometric prior: it is not a pair-distance function, and it is sensitive to cluster boundaries. The authors acknowledge this and make no claim of physically correct asymptotics. The paper also states honestly that "long-range" here only means "global communication without a cutoff."
Methodological Rigor
The formulation is clean: invariance and permutation proofs, a complexity analysis that admits the O(N·M·Dq·Dv) intermediate memory, and a rank bound. The matched 5M ablation is the best part of the design. It shares the backbone, charge/spin conditioning, repulsion, training recipe and 12-epoch budget, and evaluates on the full 201,865-structure validation set.
Notable gaps:
Potential Impact
The headline result is strong: a 20M model beats the 291M UMA-S on OPoly26 by 27.6% on forces and by 2–11% on energy. The force margin is real, though the energy margin over UMA-S-1.2.1 is small (1.7% total energy). Much of the gain may come from dataset specialization rather than RKA. The mechanism could be adopted as a lightweight add-on to other equivariant backbones, and it is cheap to implement. It has no demonstrated benefit outside OPoly26, no MD stability tests, and no downstream property predictions. The centroid dependence breaks size-extensivity and separability, which limits use in simulations of extended or periodic systems, and the authors concede this. The long-term impact is therefore likely to be niche: useful for finite-cluster benchmarks (OPoly26 leaderboard-style work) but unlikely to become a standard primitive for MLFFs.
Timeliness
Long-range interactions in MLIPs are a live topic (Ewald message passing, LES, EFA, AllScAIP). OPoly26 is new and the polymer MLFF space is just opening, so strong early results on it get attention. Yet the centroid-based approach is not obviously physics-aligned, and the field is moving toward physically principled long-range terms.
Strengths
Limitations
Overall
A competent, well-scoped incremental method paper with impressive benchmark numbers on a new dataset but limited demonstrated generality and mechanism validation. Impact is likely moderate and concentrated in polymer and cluster MLFF work.
Generated Nov 2, 2026
Comparison History (0)
Simple, clean global-attention add-on with strong OPoly26 numbers versus a much larger UMA, but single-seed, single-dataset, no mechanism ablations, and a centroid-based design with acknowledged physical limitations.