Whispers in the Noise: Surrogate-Guided Concept Awakening via a Multi-Agent Framework
Mengyu Sun, Ziyuan Yang, Zunlong Zhou, Junxu Liu, Haibo Hu, Yi Zhang
Abstract
Diffusion models (DMs) are widely used for text-to-image generation, but their strong generative capabilities also raise concerns about unsafe or undesirable content. Concept erasure aims to mitigate these risks by removing specific concepts from pretrained models. However, recent studies show that such methods often suppress rather than fully eliminate target concepts, leaving models vulnerable to awakening attacks. Existing approaches primarily rely on white-box access through optimization or inversion, while concept awakening under black-box constraints remains underexplored. In this work, we revisit the denoising process from a trajectory perspective and show that concept erasure mainly disrupts early-stage text-semantic alignment but does not fully prevent semantic information from propagating along the denoising dynamics. As generation proceeds, the model increasingly depends on the evolving noisy state rather than textual conditions, which creates an opportunity to bypass erased mappings. Motivated by this observation, we propose ConceptAgent, a training-free, black-box, multi-agent framework that awakens erased concepts by initializing the denoising trajectory from surrogate-guided noisy states. Extensive experiments demonstrate that ConceptAgent enables accurate and controllable awakening of erased concepts under black-box settings without access to model parameters, gradients, or internal representations. These results highlight fundamental limitations of current concept erasure methods and provide new insights into the dynamic nature of semantic control in DMs.
AI Impact Assessments
(1 model)Scientific Impact Assessment: "Whispers in the Noise: Surrogate-Guided Concept Awakening via a Multi-Agent Framework"
1. Core Contribution
This paper identifies a fundamental vulnerability in concept erasure methods for diffusion models (DMs) and proposes ConceptAgent, a black-box, training-free, multi-agent framework to exploit it. The key insight is that concept erasure primarily disrupts early-stage text-semantic alignment during denoising, while later stages—where generation becomes increasingly state-driven rather than text-driven—remain largely intact. By injecting surrogate-guided structured noise into intermediate denoising states, the method bypasses erased text-to-concept mappings without requiring model parameters, gradients, or internal representations.
The trajectory-based perspective is the paper's most meaningful conceptual contribution. The decomposition of the denoising process into a "text-conditioned estimate" and a "semantic-noise estimate" (Eq. 2), with the observation that their relative dominance shifts across timesteps, provides a useful analytical framework. Theorems 3.1 and 3.2 formalize the notions of "semantic dominance transition" and "trajectory intersection," respectively, offering theoretical grounding for why surrogate-guided states can lead to awakening of erased concepts.
2. Methodological Rigor
Theoretical analysis: The two theorems are stated but not rigorously proven in the main text. Theorem 3.1 essentially states that when the text-conditioned component dominates, text controls semantics, and vice versa—this is somewhat tautological without quantitative characterization of the transition point t*. Theorem 3.2 claims that non-injective transition operators allow trajectory intersection, which is geometrically intuitive but lacks formal proof regarding when and under what conditions such intersections occur in practice. The theoretical claims would benefit from tighter bounds and explicit assumptions about the model class.
Empirical validation: The controlled experiments in Section 4 (Figure 2) are well-designed and provide convincing evidence for the trajectory-based vulnerability. The three experimental conditions—late-stage text removal, mid-trajectory model switching, and early-stage erasure—effectively decompose the role of text conditioning across denoising stages.
Evaluation methodology: The use of ACC (ResNet-50 classification) and CLIP score as metrics is standard but limited. ACC measures whether the classifier recognizes the target concept, but doesn't capture visual quality, diversity, or subtlety of awakening. The evaluation across three erasure methods (UCE, RECE, SPEED) and three MLLMs (GPT-5, Gemini 3.0, Qwen-7B) provides reasonable breadth. However, only four target concepts are tested (golf ball, tench, garbage truck, pistol), which is a narrow evaluation set that limits conclusions about generalizability.
Reproducibility concerns: Several design choices appear manually tuned without clear justification—tf=70 steps for noise injection, tcan=35 for composition refinement, K=100 surrogates, J=3 physical refinement prompts. The sensitivity to these hyperparameters is not analyzed.
3. Potential Impact
AI Safety: The paper's primary impact lies in exposing vulnerabilities in concept erasure—a critical safety mechanism for deployed DMs. Demonstrating that black-box attacks can awaken erased concepts (including harmful ones like nudity and blood, as shown in Figure 8) is an important finding for the safety community. This could motivate development of more robust erasure techniques that account for trajectory-level dynamics.
Defense design: The trajectory-based perspective suggests that future erasure methods should not only disrupt text-concept mappings but also address semantic propagation through intermediate states. This is a constructive insight that could reshape erasure method design.
Dual-use concerns: The paper demonstrates awakening of safety-critical concepts (pistol, nudity, blood) under black-box constraints, which raises dual-use concerns. While the authors frame this as exposing vulnerabilities, the detailed methodology and promised code release lower the barrier for malicious use.
4. Timeliness & Relevance
The paper addresses a timely concern. Concept erasure has become a standard safety mechanism for deployed diffusion models, and understanding its limitations is urgent. The black-box setting is particularly relevant since most users interact with DMs through APIs without model access. Prior awakening attacks largely assumed white-box access, making this work fill a genuine gap.
The use of modern MLLMs (GPT-5, Gemini 3.0) as agent backbones is timely, leveraging the reasoning capabilities of frontier models for structured adversarial attacks—a paradigm that is likely to become increasingly relevant.
5. Strengths & Limitations
Strengths:
Limitations:
Additional Observations
The paper's framing as a "multi-agent" system, while trendy, is somewhat superficial—the agents are essentially pipeline stages orchestrated by MLLM prompting. The multi-agent terminology adds complexity without clear methodological benefit over a standard pipeline design.
The comparison baseline set is thin. More recent awakening methods should be compared, and the comparison with ARC and CCE lacks controlled experimental conditions (e.g., ARC is tested only under SPEED erasure in Figure 6).
Generated May 19, 2026
Comparison History (19)
Paper 1 addresses a fundamental limitation in diffusion model safety—showing concept erasure methods are inherently flawed by analyzing denoising trajectories. Its black-box, training-free attack framework (ConceptAgent) has broader implications for AI safety policy and model deployment. The theoretical insight about early-stage disruption vs. later semantic propagation is novel and generalizable. Paper 2, while practically valuable for LLM agent security, addresses a narrower problem (memory poisoning auditing) with a more incremental contribution combining existing techniques (counterfactual analysis + anomaly detection). Paper 1's findings challenge foundational assumptions in content safety, likely spurring broader research.
Paper 1 addresses a fundamental security vulnerability in diffusion models—showing that concept erasure methods merely suppress rather than eliminate concepts, and demonstrating a novel black-box attack framework (ConceptAgent) that exploits denoising trajectory dynamics. This has broader impact across AI safety, content moderation, and generative model governance. The theoretical insight about early-stage text-semantic alignment disruption is significant. Paper 2, while technically solid for game AI/NPC behavior, addresses a narrower application domain with less cross-field impact. AI safety research currently commands greater urgency and broader scientific attention.
Paper 1 presents a fundamental advancement in efficient AI reasoning by enabling tiny models (7M parameters) to outperform massive frontier LLMs on complex tasks using test-time compute scaling. This challenges the current paradigm of relying solely on massive scale for reasoning, offering immense potential for edge computing, democratizing AI, and inspiring alternative architectures. Paper 2, while offering important insights into AI safety and diffusion model vulnerabilities, has a narrower scope focused on adversarial attacks against concept erasure.
Paper 2 addresses an urgent and highly topical issue: the safety and security of widely deployed generative AI models. By exposing black-box vulnerabilities in concept erasure mechanisms, it has immediate, real-world implications for AI deployment and red-teaming. While Paper 1 offers a strong interdisciplinary bridge, the explosive growth and focus on foundation model safety gives Paper 2 a higher potential for rapid, widespread scientific impact.
Paper 1 (CardioThink) addresses a critical gap in clinical AI by introducing structured physician-inspired reasoning for ECG diagnosis, combining interpretability with accuracy. Its novel SSPO optimization method and clinically aligned multi-stage reasoning framework have broad implications for trustworthy medical AI beyond just ECG. Paper 2 (ConceptAgent) makes a solid contribution to AI safety by exposing limitations of concept erasure in diffusion models, but its impact is more niche—primarily relevant to the adversarial robustness of generative models. Paper 1's potential for real-world clinical deployment and cross-domain applicability in medical AI gives it higher overall impact.
Paper 2 is likely to have higher impact due to timeliness and broad relevance: it addresses safety and robustness of diffusion models, a high-velocity area with immediate implications for deployment, governance, and adversarial ML. The proposed training-free, black-box multi-agent attack framework is a novel angle and can influence both defensive concept-erasure research and evaluation standards across generative models. Paper 1 is methodologically rigorous and valuable for planning/representation learning, but its impact is narrower and in a more mature subfield, with fewer cross-domain and near-term real-world stakes.
Paper 2 addresses a critical bottleneck in aligning Large Language Models for complex reasoning: token-level credit assignment in reinforcement learning. By introducing a novel self-distillation mechanism with a reflection bottleneck, it tackles the timely and highly impactful problem of improving LLM reasoning capabilities without late-stage collapse. While Paper 1 offers valuable insights into diffusion model vulnerabilities (AI safety), advancing foundational LLM reasoning (Paper 2) currently commands broader applicability and higher transformative potential across diverse domains such as mathematics, science, and general tool-use.
Paper 2 demonstrates significant breadth of impact and potential for real-world applications by accelerating scientific discovery across multiple disciplines (physics, chemistry, biology, materials science). Its inclusion of theoretical guarantees alongside a real-world wet-lab experiment (battery electrolytes) showcases strong methodological rigor. While Paper 1 addresses an important AI safety vulnerability, Paper 2's potential to broadly enhance empirical research optimization gives it a higher overall scientific impact.
Paper 1 addresses a critical vulnerability in AI safety by revealing fundamental flaws in concept erasure for diffusion models. Its novel, black-box approach provides deep theoretical insights into denoising dynamics and semantic control. In contrast, Paper 2 is an empirical system-integration study of existing LLM agent paradigms. While practically useful for software engineering, Paper 1 offers significantly higher theoretical novelty, methodological innovation, and broader implications for the rapidly growing field of generative AI alignment and security.
Paper 2 addresses the critical field of LLM reasoning and mechanistic interpretability. By introducing the 'polylogue' to dynamically monitor and steer latent persona directions during generation, it provides a novel, interpretable approach to reasoning-time control. This has broader implications for improving LLM accuracy, alignment, and safety across numerous NLP applications, giving it a higher potential impact than Paper 1's narrower focus on bypassing concept erasure in diffusion models.