The Arbiter Agent: Continually Monitoring Multi-Agent Conversations to Detect Emergent Misalignment
Filippo Tonini, Federico Torrielli, Anton Danholt Lautrup, Peter Schneider-Kamp, Mustafa Mert Çelikok, Lukas Galke Poech
Abstract
As AI systems built from multiple language-model agents become more common, they are increasingly used to make decisions together: discussing, negotiating, and acting on shared tasks. While individual agents may appear well-aligned when tested on their own, problems can arise from how they interact with one another. We introduce the Arbiter, an agent designed to monitor multi-agent conversations in real time and identify which participants may be behaving in misaligned ways. The Arbiter operates under a limited "inspection budget", meaning it must decide carefully how to use its resources. As it observes a conversation step by step, it can choose to wait, question a participant, examine internal information such as system prompts or reasoning traces, or log concerning behavior. At the end, it produces a report identifying the likely source of misalignment. We evaluate the Arbiter across five conversation conditions, ranging from risky financial advice model organisms to evaluation-aware and colluding agents, we test five tool configurations of increasing capability and two backbone models. We find that the Arbiter reliably detects misaligned agents well before the end of the conversation, with active inspection tools improving both detection accuracy and speed. Weight-induced misalignment proves hardest to detect, while instruction-induced misalignment is identified reliably even under passive observation. The logging tool exhibits a dual effect, improving recall at the cost of precision. These results suggest that continual, budget-aware monitoring can effectively catch misalignment, and that overseeing multi-agent systems may require treating the auditor as an active participant in the process. The code is available at https://github.com/aisilab/arbiter.
AI Impact Assessments
(1 models)Scientific Impact Assessment: The Arbiter Agent
1. Core Contribution
The paper introduces the Arbiter, a budget-constrained monitoring agent that observes multi-agent LLM conversations in real time and identifies misaligned participants. The key novelty lies in the combination of three design choices: (1) continual monitoring rather than post-hoc analysis, (2) a toolkit of active inspection capabilities (interrogation, CoT inspection, system prompt inspection, incident logging), and (3) an explicit budget mechanism that forces the monitor to allocate inspection resources strategically. The Arbiter operates as a ReAct-style reasoning loop, selecting tools at each conversational turn.
The problem addressed—detecting misalignment in multi-agent systems where individually aligned agents may produce system-level failures—is genuine and underexplored. Most prior work focuses on single-model auditing or post-hoc evaluation. The extension to continual, multi-agent monitoring is a meaningful conceptual step.
2. Methodological Rigor
The experimental design has both strengths and notable weaknesses. On the positive side, the authors systematically vary two independent dimensions (conversation conditions × tool configurations), test five misalignment scenarios, and include a no-misalignment baseline for false positive measurement. The 20 replications per cell with reported SEMs provide some statistical grounding.
However, several methodological concerns limit confidence in the results:
The finding that weight-induced misalignment is hardest to detect (F1=0.12 under pure observation, 0.51 with tools) is informative but also reveals a significant limitation—the system performs poorly on arguably the most realistic and dangerous form of misalignment. The collusion scenario achieving perfect F1 across all configurations seems to reflect the overtness of the steganographic signals rather than genuine detection capability.
3. Potential Impact
The paper addresses a timely concern in AI safety. As multi-agent systems become more prevalent in production (e.g., multi-agent coding assistants, collaborative decision-making systems), the need for runtime monitoring will grow. The Arbiter framework provides a useful conceptual scaffold:
The open-source release enhances potential for adoption and extension. However, the gap between the current evaluation (toy scenarios, weak monitored agents) and real deployment scenarios is substantial. The paper is better understood as a proof-of-concept than a deployable system.
4. Timeliness & Relevance
The paper is well-timed. Multi-agent AI systems are rapidly proliferating, and the AI safety community has identified emergent misalignment as a growing concern (Betley et al. 2026, Carichon et al. 2025). The paper connects to and extends the AI control framework of Greenblatt et al. (2024) and the SHADE-Arena benchmarks of Kutasov et al. (2026), positioning itself at the intersection of alignment monitoring and multi-agent safety.
The concern about CoT faithfulness erosion under training (Korbak et al. 2025) and monitor-aware sandbagging (Li et al. 2025) adds urgency to the question of whether active monitoring can remain effective as models become more capable—a question this paper begins to address with its evaluation-aware condition.
5. Strengths & Limitations
Key Strengths:
Notable Limitations:
Missing Comparisons: The paper would benefit from comparisons against simpler baselines (e.g., a post-hoc judge reading the full transcript, or a fixed-rule classifier) to quantify the value of the agentic, continual monitoring approach.
Summary
This paper makes a meaningful conceptual contribution by formalizing budget-constrained, continual monitoring of multi-agent conversations for misalignment detection. The empirical findings, while preliminary, reveal useful patterns about tool utility and failure modes. However, the evaluation's limited scale, capability asymmetry, and narrow misalignment scenarios constrain the strength of its conclusions. It is best viewed as an early-stage framework paper that opens a research direction rather than providing definitive answers about the feasibility of multi-agent oversight.
Generated Jun 10, 2026
Comparison History (17)
Paper 1 targets a broadly relevant and timely problem—safety/oversight of emerging multi-agent LLM systems—introducing a budget-aware, interactive monitoring paradigm with evaluations spanning multiple adversarial and realistic conditions. Its approach can generalize across domains and has potential impact on alignment, security, and deployment governance for agentic AI. Paper 2 is methodologically substantial and clinically relevant, but its impact is narrower (pulmonology-specific KG/LLM adaptation) and faces higher translational barriers (clinical validation, regulation, data/site shift). Overall breadth and timeliness favor Paper 1.
Paper 1 targets an urgent, under-addressed safety gap: emergent misalignment in multi-agent LLM systems, proposing a continual, budget-aware auditor that actively inspects conversations. This is both novel and timely as agentic systems proliferate, with clear real-world applicability (deployment monitoring, governance, incident detection) and potential cross-field impact (AI safety, HCI, security, multi-agent systems). The evaluation spans multiple adversarial conditions and tool configurations, suggesting solid rigor. Paper 2 is a meaningful RL exploration refinement, but is narrower in scope and likely incremental relative to the broader, high-stakes monitoring framework in Paper 1.
Paper 1 tackles the highly relevant and complex problem of emergent misalignment in multi-agent systems. Its introduction of an active, budget-aware auditing agent provides a novel methodological framework for dynamic AI oversight, extending beyond static evaluation. While Paper 2 offers a valuable benchmark, Paper 1's proactive approach addresses critical real-world safety challenges in deploying autonomous, interacting AI agents, suggesting broader theoretical and practical implications for AI alignment.
Paper 1 tackles a fundamental bottleneck in multi-modal AI by removing the need for paired data during knowledge distillation. Its strong theoretical foundations, combined with practical algorithms, offer broad applicability across diverse modalities. While Paper 2 addresses a highly timely issue in AI safety, Paper 1's foundational contributions to representation learning and model efficiency give it a higher potential for widespread, lasting scientific and practical impact across multiple domains.
Paper 2 addresses a critically timely and broadly impactful topic—the environmental footprint of AI-driven hyperscale data centers—with a rigorous empirical methodology covering 403 facilities. Its findings (e.g., HDC carbon intensity 48% above grid average) have immediate policy implications and relevance across environmental science, energy policy, computer science, and industry. Paper 1 presents a novel AI safety monitoring approach but targets a narrower, more speculative research community. Paper 2's quantitative, data-driven assessment fills a significant knowledge gap with broad cross-disciplinary relevance and real-world policy urgency.
Paper 1 tackles a critical and emerging challenge in AI safety: detecting emergent misalignment in multi-agent systems. Its novel approach of using an active, budget-aware 'Arbiter' agent to monitor conversations in real-time offers significant contributions to AI alignment and oversight. While Paper 2 provides a valuable architectural improvement for LLM memory, Paper 1 addresses a fundamental safety bottleneck for the real-world deployment of autonomous AI, likely leading to broader foundational impact across the field.
Paper 1 addresses the increasingly critical problem of AI safety in multi-agent systems, which is highly timely as agentic AI deployments scale. Its novel contribution—a budget-aware monitoring agent that actively inspects multi-agent conversations for emergent misalignment—has broad applicability across any domain using multi-agent LLM systems. The methodology is rigorous with systematic evaluation across multiple conditions and configurations. Paper 2 addresses a narrower domain (supply chain) with a complex but specialized framework. While technically sophisticated, its impact is more constrained. Paper 1's relevance to AI safety gives it broader cross-field significance and timeliness.
Paper 2 likely has higher scientific impact due to stronger timeliness and cross-field relevance: monitoring and auditing multi-agent systems targets a growing deployment paradigm and connects to AI safety, governance, and security. The “budget-aware, continual” oversight framing and active inspection actions provide a generally applicable methodology with clear real-world applications (enterprise agents, decision support, high-stakes domains). Paper 1 is technically novel and useful for code-to-visual generation quality, but its impact is narrower (visual artifact rendering) and more application-specific, whereas Paper 2’s ideas can generalize across many multi-agent settings.
Paper 2 addresses the increasingly critical problem of AI safety in multi-agent systems, a rapidly growing area with broad implications across AI deployment. Its novel 'Arbiter' framework for real-time, budget-aware monitoring of emergent misalignment is timely and has immediate practical applications as multi-agent systems proliferate. Paper 1, while methodologically sound, addresses a narrower problem (LLM-based math proof verification) with more limited applicability. Paper 2's contributions to AI safety, oversight mechanisms, and multi-agent alignment have broader cross-field impact and greater urgency given current AI deployment trends.
Paper 2 addresses the critical and timely problem of AI safety in multi-agent systems, which has broad implications as LLM-based agent systems proliferate. It introduces a novel monitoring framework (the Arbiter) with a principled budget-aware inspection approach, tested across multiple misalignment conditions. This work has wider cross-disciplinary impact (AI safety, governance, multi-agent systems) and addresses a growing concern in the AI community. Paper 1, while technically solid, is more narrowly focused on architectural floor plan furnishing with a relatively small dataset (270 plans), limiting its broader scientific impact.