When Do Local Score Models Extrapolate Across Size? A Diagnostic Theory and Benchmark

Scientific Impact Assessment

Core Contribution

This paper addresses a fundamental question in scientific generative modeling: under what conditions can score-based diffusion models with local (finite receptive field) architectures, trained on small systems, reliably generate samples from larger systems? The central insight is that architectural locality (e.g., translation-invariant CNNs) is necessary but insufficient for stable size extrapolation. Instead, the governing factor is the quasi-locality of the Gaussian-smoothed score, which is determined by the posterior covariance structure under Tweedie's formula rather than the clean interaction graph.

The key equation linking smoothed-score response to posterior correlations (Eq. 9) — showing that the score Jacobian ∂s_{L,σ}(x)_i/∂x_j = σ^{-4} Cov_{π^x}(X_{0,i}, X_{0,j}) — serves as the conceptual hinge. This elegantly explains why short-range clean interactions (like nearest-neighbor Ising) can produce long-range smoothed-score dependencies near critical points, where posterior correlations diverge.

The paper makes three concrete contributions: (1) formalizing the response-tail diagnostic, (2) proving a size-uniform local-marginal comparison theorem (Theorem 2), and (3) introducing the FDLF benchmark with exact scores and controllable response ranges.

Methodological Rigor

Theoretical framework: The size-uniform comparison theorem (Theorem 2) is carefully constructed. The proof strategy — using backward Kolmogorov equations, Itô's formula along the learned process, and a dual-generator comparison — is mathematically sound. The key assumptions (dynamic quasi-locality, initial consistency, weighted on-rollout drift error) are physically motivated and clearly stated. The theorem is deliberately local, controlling bounded-Lipschitz distance on fixed patches uniformly in system size, which is the appropriate metric for thermodynamic-limit questions.

However, the theorem is *conditional*: it assumes dynamic quasi-locality (Assumption 4) of the exact reverse process, which is itself a strong condition whose verification for specific systems is non-trivial. The paper acknowledges this honestly — it is a comparison theorem rather than an unconditional guarantee.

Benchmark design (FDLF): The benchmark is well-motivated. Using normalizing-flow-based teachers with exact densities and scores addresses a genuine gap: existing benchmarks cannot isolate size-extrapolation mechanisms because exact scores are unavailable. The three-way diagnostic (positive control, receptive-field-limited, controlled failure) is a clean experimental design.

Experiments: The 2D continuous experiments (Figures 1-2) convincingly demonstrate the predicted three-regime behavior. The receptive-field sweep showing medium-range teachers improving with larger CNN radius while long-range stress teachers do not is particularly compelling. The Ising critical-point stress test (Figure 4) provides a physically grounded demonstration of the failure mechanism. However, the experimental scale is relatively modest (up to L=64 for 2D), and the paper uses only simple CNN architectures without testing more modern or practical architectures (transformers, message-passing networks with adaptive radii).

Potential Impact

Scientific generative modeling: The paper provides actionable diagnostic criteria for practitioners building local generative models for molecular dynamics, materials science, or lattice field theory. The message — check whether your model's receptive field covers the smoothed-score response range, not just the clean interaction range — is both practical and non-obvious.

Architecture design guidance: The connection between posterior correlation length and required receptive field provides principled guidance for choosing model depth/width in size-transfer applications, rather than relying on trial-and-error.

Benchmark utility: FDLF fills a diagnostic gap. While not meant for realism, it offers a controlled testbed that any proposed size-transfer method should pass. This could become a standard sanity check.

Limitations on broader impact: The paper primarily addresses lattice systems with translation invariance. Extension to irregular geometries (molecules, proteins) is not straightforward. The connection to practical scientific applications (where one might use GNNs on molecular graphs) remains indirect.

Timeliness & Relevance

This paper is highly timely. Size transfer is a critical bottleneck in applying diffusion models to scientific problems — training on small molecules or lattices and deploying on larger ones. The rapid adoption of diffusion models in molecular generation (GeoDiff, torsional diffusion) and materials science makes this theoretical grounding valuable. The paper connects to active areas: score-based convergence theory, spatial mixing in statistical mechanics, and neural operator generalization.

Strengths

1. Clean conceptual contribution: The Tweedie-covariance link (Eq. 9) is simple, powerful, and likely to become a standard reference point for understanding locality in diffusion models.

2. Falsifiable predictions: The paper makes specific, testable predictions about when size transfer should succeed or fail, and then tests them systematically.

3. Principled benchmark: FDLF addresses a genuine methodological gap with exact ground truth.

4. Physical insight: The Ising critical-point example beautifully illustrates how phase transitions in the posterior, not the prior, determine extrapolation difficulty.

5. Honest framing: The paper is careful about what it claims and what it doesn't (conditional theorem, diagnostic rather than realism benchmark).

Limitations

1. Architectural scope: Only pure CNNs are tested. Modern scientific generative models use graph neural networks, transformers, or hybrid architectures. The paper mentions sliding-attention results in passing but doesn't present them.

2. Scale of experiments: Lattice sizes up to 64×64 and training for 1500 steps are modest. Practical scientific applications involve much larger systems and more complex distributions.

3. Gap between theory and practice: The comparison theorem requires verifying dynamic quasi-locality, which may be as difficult as the original problem for realistic systems. The theorem's utility is primarily conceptual rather than directly applicable.

4. Single-author limitations: The benchmark, while well-designed, would benefit from community validation and extension to more diverse settings.

5. No guidance on fixing failures: When size transfer fails (long-range response), the paper diagnoses the problem but doesn't propose solutions (e.g., multi-scale architectures, hierarchical approaches).

6. Discrete variable handling: The paper explicitly avoids learning scores for discrete variables, limiting applicability to mixed systems common in scientific modeling.

Overall Assessment

This is a theoretically clean and experimentally well-designed paper that identifies and formalizes an important mechanism governing size extrapolation in local score-based models. The central insight connecting smoothed-score locality to posterior covariance is both novel and practically relevant. The FDLF benchmark is a useful methodological contribution. The main limitations are the narrow architectural scope of experiments and the gap between the conditional theoretical guarantee and practical verification. The paper should influence how the community thinks about and evaluates size transfer in scientific generative modeling.