Hanzhi Liu, Chaofan Shou, Xiaonan Liu, Hongbo Wen, Yanju Chen, Ryan Jingyang Fang, Yu Feng
LLM agents have begun to find real security vulnerabilities that human auditors and automated fuzzers missed for decades, in source-available targets where the analyst can build and instrument the code. In practice the work is split among several agents, wired together by a harness: the program that fixes which roles exist, how they pass information, which tools each may call, and how retries are coordinated. When the language model is held fixed, changing only the harness can still change success rates by several-fold on public agent benchmarks, yet most harnesses are written by hand; recent harness optimizers each search only a narrow slice of the design space and rely on coarse pass/fail feedback that gives no diagnostic signal about why a trial failed. AgentFlow addresses both limitations with a typed graph DSL whose search space jointly covers agent roles, prompts, tools, communication topology, and coordination protocol, paired with a feedback-driven outer loop that reads runtime signals from the target program itself to diagnose which part of the harness caused the failure and rewrite it accordingly. We evaluate AgentFlow on TerminalBench-2 with Claude Opus 4.6 and on Google Chrome with Kimi K2.5. AgentFlow reaches 84.3% on TerminalBench-2, the highest score in the public leaderboard snapshot we evaluate against, and discovers ten previously unknown zero-day vulnerabilities in Google Chrome, including two Critical sandbox-escape vulnerabilities (CVE-2026-5280 and CVE-2026-6297).
AgentFlow introduces two interconnected innovations for automating multi-agent system design: (1) a typed graph DSL that unifies all five harness dimensions (agent roles, communication topology, message schemas, tool bindings, coordination protocol) into a single searchable representation, and (2) a feedback-driven outer optimization loop that consumes structured runtime signals (coverage maps, sanitizer reports, stderr) from the target program to diagnose *why* a harness failed, not merely *that* it failed. The key insight is that prior harness optimizers (Meta-Harness, ADAS, AFlow, MaAS) each search only a subset of the design space and rely on coarse pass/fail feedback, which the authors convincingly argue limits their ability to discover cross-component improvements.
The formalization as a five-component tuple H=(A,G,Σ,Φ,Ψ) is clean and provides a useful lens for comparing prior work (Table 3), revealing that no existing system searches all five dimensions simultaneously.
Strengths in formalization: The typed DSL is well-specified with formal syntax (Figure 2), well-formedness rules (T-Agent, T-Edge, T-Conn, T-Pipe), and a soundness proposition (4.1) ensuring no agent is dispatched with unbound template variables. The type system serves as a practical budget guard—rejecting ~20% of malformed proposals before expensive LLM evaluation—rather than claiming deep type-theoretic novelty, which is appropriately modest.
Evaluation concerns: The TerminalBench-2 evaluation follows the public leaderboard protocol, which aids comparability. However, several methodological issues warrant scrutiny:
Chrome campaign: The switch from Claude Opus 4.6 to Kimi K2.5 for the Chrome campaign introduces a confound—we cannot separate model effects from domain effects. The authors acknowledge this is a "case study" rather than a controlled comparison, which is appropriate but limits the scientific claims.
Immediate practical impact is very high. Ten confirmed zero-day vulnerabilities in Google Chrome, including two Critical sandbox escapes, represents genuinely impactful security research. These are externally validated through Chrome VRP—not self-reported benchmarks. The two Critical CVEs (CVE-2026-5280 and CVE-2026-6297) enabling sandbox escape are among the most severe vulnerability classes in browser security.
Broader methodological impact: The framing of harness design as a search problem over a typed program space is potentially influential beyond security. Any domain using multi-agent LLM systems (code generation, scientific discovery, data analysis) faces the same harness engineering challenge. The DSL abstraction and feedback-driven optimization loop are domain-agnostic in principle, with only the score function S and feedback channels O being domain-specific.
For the security community specifically: This work demonstrates that LLM-driven vulnerability discovery at scale (35M+ lines of C/C++) is feasible with mid-tier open-weight models (Kimi K2.5) when the orchestration is sufficiently sophisticated—a potentially transformative finding for both offensive and defensive security.
This paper arrives at a critical juncture. LLM agents for security (Big Sleep, Claude Mythos) have demonstrated capability but remain hand-engineered. The agent-systems community has produced several harness optimizers (ADAS, AFlow, MaAS, Meta-Harness) in 2024-2025, but none addresses the full design space. AgentFlow's timing—synthesizing these threads into a unified framework with real-world validation—is excellent. The demonstration that harness design alone accounts for a 4× performance range on identical models and tasks (cited from TerminalBench-2) makes the automation of harness design an urgent practical need.
The paper's ethical handling is commendable—responsible disclosure, withholding exploit details, and releasing only the optimizer framework. The synthesized harness topologies (Figures 6, 7) are complex enough to suggest they would be difficult to design by hand, supporting the automation thesis. The observation that prompt edits account for the largest ablation drop (-32.5pp) while structural edits contribute less (-7.9pp) is nuanced and somewhat undercuts the headline contribution of the DSL for structural search, though the authors rightly note these are complementary.
Generated Apr 23, 2026
GPUBreach demonstrates a fundamentally new class of security vulnerability—GPU-side privilege escalation via Rowhammer that crosses the GPU-CPU boundary, defeating IOMMU protections and achieving root shell access. This opens an entirely new attack surface with broad implications for GPU security architecture, cloud computing, and hardware design. While Paper 2's Chrome zero-days are impressive practical results, its contribution is primarily engineering (harness optimization for LLM agents), building on existing agent frameworks. GPUBreach's novelty in bridging GPU Rowhammer to full system compromise represents a deeper, more foundational security insight with lasting architectural implications.
While Paper 1 presents a robust framework for AI agent safety, Paper 2 demonstrates exceptional real-world impact by discovering ten zero-day vulnerabilities in Google Chrome, including two critical sandbox escapes. Paper 2's introduction of a typed graph DSL and feedback-driven optimization loop represents a major leap in automated vulnerability discovery. The tangible results (CVEs in a major browser) and state-of-the-art benchmark performance will drive massive immediate attention, industry adoption, and citations across cybersecurity and artificial intelligence communities, giving it significantly higher scientific and practical impact.
Paper 2 demonstrates profound real-world impact by discovering ten previously unknown zero-day vulnerabilities in Google Chrome, including two critical CVEs. While Paper 1 offers strong theoretical and empirical advancements in blockchain scalability, Paper 2 tackles a highly timely problem (LLM agents for cybersecurity) with an innovative feedback-driven synthesis approach. Automatically discovering critical vulnerabilities in heavily audited, widely-used software represents a massive breakthrough with broad implications for both software security and AI-driven automated reasoning. This exceptional concrete evidence of real-world applicability gives Paper 2 a substantially higher potential scientific impact.
Paper 2 introduces a fundamentally novel attack vector by exploiting hardware-specific floating-point divergence to trigger LLM backdoors. While Paper 1 shows exceptional practical results (Chrome zero-days) via agent optimization, Paper 2 exposes a foundational blind spot in AI auditing and model supply chains. This conceptual leap—using hardware math variations as an input-independent trigger—creates an entirely new subfield spanning AI security, robustness, and hardware-software co-design, indicating a broader and more enduring scientific impact.
Paper 2 demonstrates higher potential scientific impact due to its concrete, verified real-world results: discovering 10 zero-day vulnerabilities in Google Chrome, including two critical sandbox escapes with assigned CVEs. This provides undeniable evidence of practical impact in cybersecurity. The AgentFlow DSL and feedback-driven harness optimization framework also introduces a novel, generalizable methodology for multi-agent system design. While Paper 1 (Bifrost) presents a solid engineering contribution combining TEE and FHE for private LLM inference with impressive speedups, it remains largely an architectural optimization with estimator-based comparisons rather than demonstrated real-world deployment. Paper 2's breadth of impact across security, AI agents, and software engineering, combined with its immediate practical significance, gives it the edge.
Paper 2 has higher impact potential: it introduces a broadly applicable, typed DSL + feedback-driven search framework for synthesizing multi-agent harnesses, and demonstrates unusually strong real-world validation by discovering multiple previously unknown Chrome zero-days (including critical sandbox escapes). This indicates direct practical utility and high timeliness for AI-assisted security auditing. While Paper 1 is novel and efficient (inference-time activation/logit biasing) and likely influential for secure code generation, its contributions are more incremental and narrower in application scope than an end-to-end system that measurably improves vulnerability discovery in flagship software.
Paper 2 demonstrates exceptional real-world impact by discovering previously unknown critical zero-day vulnerabilities in a major software project (Google Chrome). Its approach to automating multi-agent LLM harnesses addresses a highly timely and critical problem in cybersecurity. While Paper 1 offers strong architectural improvements for FHE, Paper 2's immediate applicability, state-of-the-art benchmark performance, and novel synthesis of LLM agents with runtime feedback represent a broader and more urgently impactful advancement across both AI and security domains.
While Paper 1 provides a valuable benchmark for AI exploitation capabilities, Paper 2 demonstrates profound real-world impact by discovering 10 previously unknown zero-day vulnerabilities in Google Chrome, including two critical sandbox escapes. The introduction of a novel, feedback-driven DSL for synthesizing agent harnesses represents a significant methodological advancement with immediate, high-stakes applications in proactive vulnerability discovery.
Paper 1 audits a privacy system deployed on billions of Apple devices, revealing that differential privacy guarantees are violated for the majority of data collection. This has immediate, massive real-world impact on user privacy and will likely force Apple to fix critical bugs. It also advances the methodology for auditing proprietary privacy systems. Paper 2 is impressive with its Chrome zero-days, but its impact is more incremental within the AI-for-security space. Paper 1's findings affect a far larger population and address fundamental trust in deployed privacy claims, with broader implications for regulatory accountability and corporate transparency.
Paper 1 provides a comprehensive systematization of knowledge (SoK) on reconstruction attacks against synthetic tabular data, contributing a taxonomy, 14 attacks across 9 methods, novel methodology for interpreting attack success, and externally validated results (winning NIST CRC). Its breadth of impact spans privacy, data governance, and policy, influencing how synthetic data is deployed across many domains. Paper 2 is impressive with real zero-day Chrome vulnerabilities, but is narrower in scope (LLM agent harness optimization for vulnerability discovery) and its methodological contributions (a DSL and feedback loop) are more incremental relative to Paper 1's foundational systematization of an increasingly critical privacy problem.