Conflict-Aware Additive Guidance for Flow Models under Compositional Rewards
Xuehui Yu, Fucheng Cai, Meiyi Wang, Xiaopeng Fan, Harold Soh
Abstract
Inference-time guided sampling steers state-of-the-art diffusion and flow models without fine-tuning by interpreting the generation process as a controllable trajectory. This provides a simple and flexible way to inject external constraints (e.g., cost functions or pre-trained verifiers) for controlled generation. However, existing methods often fail when composing multiple constraints simultaneously, which leads to deviations from the true data manifold. In this work, we identify root causes of this off-manifold drift and find that the approximation error scales severely with gradient misalignment. Building on these findings, we propose Conflict-Aware Additive Guidance (), a lightweight and learnable method, which actively rectifies off-manifold drift by dynamically detecting and resolving gradient conflicts. We validate across diverse domains, ranging from synthetic datasets and image editing to generative decision-making for planning and control. Our results demonstrate that effectively rectifies off-manifold drift, surpassing baselines in generation fidelity while using light compute. Code is available at https://github.com/yuxuehui/CAR-guidance.
AI Impact Assessments
(1 model)Scientific Impact Assessment: Conflict-Aware Additive Guidance for Flow Models under Compositional Rewards
1. Core Contribution
This paper tackles a well-defined and practically important problem: when steering pre-trained flow models at inference time using multiple reward functions simultaneously, approximate guidance methods suffer from off-manifold drift due to gradient conflicts between competing objectives. The authors make two main contributions:
Theoretical: They derive an upper bound on the approximation error of guided sampling (Theorem 4.2), decomposing it into three interpretable terms: coupling shift error, gradient misalignment error, and localized approximation error. The key insight is that error scales with G(G-1) and (1-cos φ), where G is the number of rewards and φ captures angular divergence between guidance gradients.
Methodological: They propose CAR guidance (g^car), which dynamically blends approximate guidance with a learned value-gradient correction, gated by a conflict-aware weight that activates only in regions of significant gradient misalignment. The learned component uses Terminal Value Regression (TVR), which avoids bootstrapping instability by regressing directly against terminal rewards — a clean insight enabled by the deterministic nature of flow ODEs.
2. Methodological Rigor
Theoretical analysis is well-structured. The three-term decomposition provides clear intuition, and the connection between gradient misalignment and "energy traps" (spurious local minima from destructive gradient interference) is geometrically compelling. The energy dissipation framework (Definition B.2) provides a clean characterization of when and why approximate guidance fails. However, some assumptions (e.g., ∥g_j^CI∥ ≈ μ for all j) are strong and may not hold in practice.
Experimental evaluation is thorough and spans diverse domains: 2D synthetic benchmarks (where ground truth is available), Maze2D planning, ManiSkill2 robotic manipulation (3D point clouds), and CelebA-HQ image editing. This breadth is a strength. The synthetic experiments provide clear visualization of the failure mode (energy traps) and the correction mechanism. The inclusion of ablation studies on components (g_ψ vs. w_t), parameterization choices (scalar V vs. vector field), and the conflict threshold τ adds credibility.
Weaknesses in rigor: The comparison with GLASS-FKS may not be entirely fair — different base models and reward compositions are acknowledged. The paper reports means and standard deviations across 5 seeds, which is reasonable but not extensive. Some experimental settings (e.g., ManiSkill2 with only 100 demonstrations) are relatively small-scale.
3. Potential Impact
The paper addresses a genuine bottleneck in deploying generative models with heterogeneous runtime constraints. The practical impact is potentially significant across several domains:
The insight that gradient conflict detection can be cheaply computed and used to gate corrections is elegant and could influence how future guidance methods are designed. The TVR training procedure is also a clean contribution that others could adopt independently.
4. Timeliness & Relevance
This work is highly timely. Flow matching and rectified flows are rapidly becoming the dominant paradigm for generative modeling, and inference-time steering is increasingly important as models scale. The compositional constraint setting is practically motivated — real-world applications rarely involve a single objective. The paper sits at the intersection of generative modeling, optimal transport theory, and multi-objective optimization, connecting these communities.
The positioning between approximate and exact guidance methods (Figure 7) is clear and fills a genuine gap. The connection to the "deadly triad" in RL and the proposed TVR solution demonstrates thoughtful cross-pollination of ideas.
5. Strengths & Limitations
Key Strengths:
Notable Limitations:
Minor observations: The paper is well-written with effective visualizations. The "energy trap" concept is intuitive and well-illustrated. The positioning as "between approximate and exact guidance" is honest and accurate.
Generated May 21, 2026
Comparison History (28)
Paper 2 addresses a critical bottleneck in LLM development: the scarcity of high-quality supervision. By advancing 'weak-to-strong' generalization through preference delta aggregation and LoRA merging, it offers a highly scalable and practical solution to the alignment problem. This has massive implications for pushing the capabilities of frontier models without requiring human-level or superior data. While Paper 1 provides a valuable methodological improvement for diffusion models, Paper 2's focus on scalable oversight for LLMs is likely to have a broader and more immediate impact across the AI research landscape.
Paper 1 addresses a timely and broadly impactful question about how AI usage affects human skill development, with implications spanning education, workforce training, and AI policy. Its findings—that heavy AI use can substitute for rather than complement learning—have immediate real-world relevance as AI tools become ubiquitous. Paper 2 makes a solid technical contribution to guided sampling in flow/diffusion models, but its impact is more narrowly scoped to the generative modeling community. Paper 1's breadth of societal impact and timeliness give it higher potential scientific impact.
Paper 1 addresses a fundamental and broadly applicable problem in guided generation across diffusion/flow models, proposing a principled solution (conflict-aware gradient correction) validated across diverse domains (images, planning, control). Its practical utility is high given the widespread adoption of diffusion models. Paper 2 introduces a valuable evaluation metric for VLM explainability, but its impact is more niche—focused on XAI evaluation methodology rather than enabling new capabilities. Paper 1's cross-domain applicability, actionable method, and alignment with the rapidly growing generative modeling field give it broader potential impact.
Paper 1 likely has higher scientific impact due to broader methodological relevance: it addresses a fundamental, widely encountered failure mode in diffusion/flow guidance under compositional constraints, offers a principled analysis (gradient misalignment → off-manifold drift), and proposes a lightweight, learnable correction applicable across multiple domains (images, synthetic, planning/control). This gives wide cross-field applicability and timeliness for controllable generative modeling. Paper 2 is impactful for EDA agents, but its scope is more domain-specific and depends on verifier setups, potentially limiting breadth despite strong practical relevance.
Paper 2 exposes a fundamental flaw in current VLM explainability evaluation and introduces a theoretically grounded, scalable metric to resolve it. Because reliable benchmarks and evaluation frameworks often redirect community efforts and are essential for high-stakes AI safety, Paper 2 is likely to have a broader foundational impact across the rapidly growing field of multimodal models compared to the algorithmic improvements in generative guidance proposed in Paper 1.
Paper 2 likely has higher scientific impact: it addresses a broadly encountered limitation of inference-time guidance (composing multiple rewards) in diffusion/flow models, provides a principled diagnosis (gradient misalignment driving off-manifold drift), and proposes a general, lightweight, learnable fix validated across multiple domains (images and decision-making/control). This combination of theoretical insight + wide applicability can influence controlled generation, alignment, and planning. Paper 1 is novel and practical for LLM agents, but is currently demonstrated mainly on deterministic agent benchmarks and depends on environment-specific harness engineering, making impact potentially narrower.
Paper 1 addresses a fundamental problem in compositional guided generation across diffusion/flow models with rigorous theoretical analysis (identifying gradient misalignment as root cause of off-manifold drift) and broad empirical validation across diverse domains (synthetic, image editing, planning/control). The theoretical insights about conflict-aware gradient composition are novel and broadly applicable. Paper 2 presents a practical engineering contribution for modular LLM specialization, but is more incremental—combining existing ideas (delta modules, compression) in a system-level framework. Paper 1's methodological depth and cross-domain generality suggest broader scientific impact.
Paper 2 addresses a fundamental challenge in guided generation across diffusion/flow models—composing multiple constraints without off-manifold drift—which is broadly applicable across images, planning, and other generative domains. Its theoretical insight (gradient misalignment causing approximation error) and lightweight, learnable solution (conflict-aware guidance) have wider cross-domain impact potential. Paper 1, while rigorous and practically valuable for autonomous driving testing, addresses a more domain-specific problem. Paper 2's generality across generative modeling paradigms gives it broader scientific influence.
Paper 2 is likely to have higher scientific impact because it introduces a principled, general-purpose evaluation framework (a family of proper scoring-rule metrics) for uncertainty-augmented systems, directly targeting high-stakes decision-making. Such metrics can be adopted broadly across ML subfields (classification, generation, selective prediction, calibration, human-AI decision support) and influence standard benchmarks and reporting practices. Its theoretical grounding (proper scoring rules) supports methodological rigor and long-term relevance. Paper 1 is a strong, timely contribution to guided generative modeling, but is more specialized and may have narrower cross-domain uptake.
Paper 2 likely has higher scientific impact: it proposes a concrete, generally applicable algorithmic improvement for inference-time control of diffusion/flow models, a highly active area with broad downstream use (image editing, planning/control, constrained generation). It identifies a clear failure mode (off-manifold drift under compositional rewards), analyzes a root cause (gradient misalignment), and introduces a lightweight, learnable fix validated across multiple domains with released code—supporting rigor, reproducibility, and adoption. Paper 1 is novel for negotiation research but is narrower in scope and its impact depends on uptake in social-science experimentation and agent design.