From Uniform to Learned Graph Priors: Diffusion for Structure Discovery

Paper 1 (Diff-prior) addresses a fundamental limitation in neural relational inference by introducing diffusion-based learnable graph priors, offering broad applicability across NRI architectures and real-world dynamical systems. It provides a novel, principled framework combining diffusion models with variational inference for structure discovery—a widely relevant problem. Paper 2 provides valuable interpretability insights about Block Attention Residuals but is narrower in scope, focused on a specific architecture variant at a single scale, and its conclusions (routing exposure is necessary but insufficient) are somewhat expected. Paper 1's methodological contribution and broader applicability suggest higher impact.

Paper 1 likely has higher impact due to strong timeliness and immediate real-world applicability: it targets a pressing systems bottleneck in LLM inference (KV reuse for RAG/agents) and offers an unusually low-intrusion design (<100 LOC) compatible with vLLM and CPU offload, with large measured throughput/latency gains. Its primitives can implement multiple PIC variants, suggesting broad adoption potential across inference stacks. Paper 2 is novel and methodologically sound for learned graph priors, but its impact is narrower (NRI benchmarks) and less immediately deployable than an inference-server optimization.

Paper 1 likely has higher scientific impact due to stronger novelty and broader methodological relevance: it introduces a diffusion-parameterized learned prior for latent graph structure discovery, a reusable idea that can transfer to many structured latent-variable models beyond NRI (e.g., causal/relational discovery, discrete variational inference). This is timely given diffusion models’ prominence and addresses a known limitation (factorized priors) with a principled calibration mechanism. Paper 2 is highly applied and well-evaluated, but largely combines established components (wavelets + Isolation Forest) and its impact is more domain-specific.

Paper 1 offers higher scientific impact due to its broader methodological innovation. While Paper 2 presents a highly effective but incremental architectural improvement (adaptive gating) for neural PDE solvers, Paper 1 introduces a novel paradigm connecting diffusion models with variational inference to learn adaptive graph priors. This addresses a fundamental limitation in Neural Relational Inference (oversimplified uniform priors) and has broad applicability across diverse fields requiring structural discovery and relational reasoning, such as biology, physics, and complex systems analysis.

Paper 2 introduces a novel and broadly applicable framework (Diff-prior) that addresses a fundamental limitation in neural relational inference by replacing simplistic uniform priors with learned diffusion-based priors for graph structure discovery. This has broader impact across multiple fields (physics simulations, biology, social networks) and advances both generative modeling and structure learning methodology. Paper 1 addresses a narrower engineering optimization problem specific to memristor-based analog computation for ASR, with more limited scope and applicability despite being technically sound.

Paper 1 has higher estimated impact due to its strong real-world applicability and timeliness: guaranteeing strict satisfaction of arbitrary polyhedral safety constraints in flow-based generative models directly targets safety-critical planning/control deployment. The projection-free, solver-free approach with zero constraint violations and reduced latency suggests substantial practical value and broad relevance across robotics, control, and constrained generative modeling. Paper 2 is innovative in using diffusion to learn adaptive graph priors for NRI and likely improves structure discovery, but its impact is more niche to relational inference benchmarks and less directly tied to deployment-critical guarantees.

Paper 2 is likely higher impact due to stronger novelty and broader relevance: introducing a diffusion-parameterized learned prior for calibrating latent graph posteriors addresses a known limitation (factorized/uniform priors) in neural relational inference and is broadly applicable to structure discovery across physical, biological, and social dynamical systems. The method is timely (diffusion for probabilistic modeling), integrates with multiple NRI-family architectures, and targets reliability/decisiveness of inferred graphs—a key real-world need. Paper 1 is useful and efficient for time-series clustering but is more niche and more incremental in combining existing components.

Paper 2 bridges the highly successful concepts of in-context learning and chain-of-thought from LLMs to neural operators for PDEs. This zero-shot generalization approach to out-of-distribution scientific machine learning tasks offers broader cross-disciplinary impact and higher novelty compared to Paper 1's algorithmic improvement of graph priors in neural relational inference.

Paper 2 likely has higher impact: it addresses a widely used foundation model (Vision Transformers) and targets transfer learning practice, supported by theory plus extensive empirical validation (1,000+ finetuning runs). Its conclusions can directly influence common finetuning strategies across many vision tasks and potentially extend to other transformer domains, giving broad, timely applicability. Paper 1 is novel but more specialized to NRI/graph structure discovery, with narrower immediate adoption and impact scope despite solid methodological innovation.

Paper 1 is likely to have higher impact: it introduces a practical, GPU-optimized, auditable ICA workflow for LLM interpretability that removes a major bottleneck (training/storing SAEs) and shows competitive-to-better results on established benchmarks across multiple prominent models. This is timely and broadly relevant to mechanistic interpretability, model debugging, and control—areas with wide cross-field interest and rapid uptake. Paper 2 is a solid methodological improvement for NRI via diffusion-learned priors, but is more niche (trajectory-to-graph discovery) and may have narrower immediate adoption outside structured latent-variable modeling.