This assessment is based on version 1 of this paper. Version 2 is now available on arXiv — the authors may have revised their methods, results, or conclusions.
ARGUS: Agentic GPU Optimization Guided by Data-Flow Invariants
Haohui Mai, Xiaoyan Guo, Xiangyun Ding, Daifeng Li, Qiuchu Yu, Chenzhun Guo, Cong Wang, Jiacheng Zhao
Abstract
LLM-based coding agents can generate functionally correct GPU kernels, yet their performance remains far below hand-optimized libraries on critical computations such as matrix multiplication, attention, and Mixture-of-Experts (MoE). Peak GPU performance requires coordinated reasoning over tightly coupled optimizations, including tiling, shared-memory staging, software pipelining, and instruction scheduling, while existing agents rely on sparse pass/fail feedback, leaving them unable to diagnose global constraint violations. We present Argus, an agentic framework that addresses this through data-flow invariants: compile-time specifications encoding how data must be choreographed throughout kernel execution. Argus introduces a tile-based, Pythonic DSL exposing hardware instructions and compiler policies while hiding low-level representations. The DSL provides tag functions to propagate symbolic annotations through data and control flow, and tag assertions to enforce relational constraints at use sites. When violations occur, the compiler returns concrete counterexamples identifying the thread, data element, and program point, enabling dense, structured feedback for targeted fixes. Invariants are verified at compile time via abstract interpretation over a layout algebra and SMT solving, with zero runtime overhead. An in-context reinforcement learning planner learns to select optimizations and synthesize effective invariants, supported by a curated knowledge base of GPU optimization techniques. We evaluate Argus on the AMD MI300X GPU across GEMM, flash attention, and MoE kernels accounting for over 90% of GPU time in LLM inference. Generated kernels achieve 99-104% of state-of-the-art hand-optimized assembly throughput and are 2-1543x faster than existing agentic systems. Argus further generalizes to 200 KernelBench tasks, solving 100% of Level 1 and 90% of Level 2 problems.
AI Impact Assessments
(3 models)Scientific Impact Assessment: ARGUS: Agentic GPU Optimization Guided by Data-Flow Invariants
1. Core Contribution
Argus addresses a fundamental gap in LLM-based GPU kernel generation: while agents can produce functionally correct kernels, they fail catastrophically at achieving competitive performance on critical workloads (GEMM, attention, MoE). The core insight is that peak GPU performance requires coordinated reasoning across tightly coupled optimizations (tiling, shared-memory staging, software pipelining, instruction scheduling), and existing agents lack the feedback mechanisms to navigate this space.
The key innovation is data-flow invariants—compile-time specifications encoding how data must be choreographed through kernel execution—adapted from information flow security to GPU optimization. This is operationalized through three synergistic mechanisms: (1) a tile-based Pythonic DSL with tag functions and tag assertions for expressing invariants, (2) static analysis via abstract interpretation over layout algebras and SMT solving for zero-overhead compile-time verification with concrete counterexamples, and (3) an in-context reinforcement learning (ICRL) planner that learns to select optimizations and synthesize invariants.
The conceptual transfer from information flow analysis (HiStar, JFlow) to GPU kernel optimization is genuinely novel and intellectually compelling.
2. Methodological Rigor
Strengths: The evaluation is thorough, covering three production kernel families (GEMM, flash attention, MoE) that account for >90% of LLM inference GPU time on AMD MI300X. The comparison against both hand-optimized libraries (HipBlasLt, HipKittens, AITER) and five agentic baselines (KernelBench, KSearch, CUDAForge, KernelFalcon) provides comprehensive context. The ablation study (Figure 2) carefully isolates contributions of each optimization level, revealing non-obvious interactions (e.g., pipelining initially degrading performance until instruction scheduling recovers gains). Table 1's systematic breakdown of optimizations across systems is particularly informative.
The invariant effectiveness study (Table 3) isolates the contribution of data-flow invariants, showing Pass@1 improvements of 15-17 percentage points with reduced token usage, providing direct evidence for the mechanism's utility.
Concerns: The evaluation targets only AMD MI300X. While the authors claim generalizability to NVIDIA GPUs, this is unverified. The knowledge base is expert-curated, which raises questions about scalability and the degree of human effort required for new workloads. The ICRL loop's convergence properties and sample efficiency are not characterized. The paper also uses GLM-5 exclusively; sensitivity to the underlying LLM is unexplored. The 10-iteration budget for agentic frameworks may disadvantage some baselines.
3. Potential Impact
Immediate impact: The results are striking—99-104% of hand-optimized assembly throughput and 2-1543× speedups over existing agentic systems. If reproducible, this could significantly reduce the months of engineering required for production GPU kernels, particularly valuable given yearly hardware refreshes and evolving workloads.
Broader implications: The data-flow invariant approach could influence how we think about LLM-guided program optimization more generally. The insight that compile-time invariant violations provide dense, structured process rewards (rather than sparse pass/fail signals) is applicable beyond GPU kernels to any domain requiring coordinated multi-level optimization. The DSL design philosophy—exposing policy while hiding mechanisms—offers a template for LLM-oriented language design.
Practical limitations: The expert-curated knowledge base is both a strength (auditability) and bottleneck (scalability). Porting to new hardware requires domain expertise for both intrinsic integration and knowledge base extension.
4. Timeliness & Relevance
This paper is exceptionally timely. The convergence of three trends creates strong demand: (1) massive GPU infrastructure investment for LLM deployment where kernel efficiency directly translates to billions in savings, (2) rapid improvement in LLM coding capabilities creating opportunity for automated kernel generation, and (3) the widening gap between hardware evolution and software optimization toolchains. The focus on AMD MI300X is strategically relevant given AMD's growing role in AI infrastructure, though also limits immediate adoption breadth.
The paper directly addresses the bottleneck that no existing agentic system could produce kernels competitive with hand-optimized libraries on the most performance-critical workloads—a gap that has limited practical adoption of LLM-based kernel generation.
5. Strengths & Limitations
Key Strengths:
Limitations:
Additional Observations
The paper makes an important architectural decision by targeting a DSL rather than raw CUDA/assembly, keeping context manageable for LLMs while maintaining expressivity. The comparison showing Argus's DSL kernel is ~1.8× faster than hand-optimized Triton for GEMM quantifies the "abstraction tax" of higher-level languages and validates the DSL design point.
The distinction between process rewards (invariant violations) and outcome rewards (runtime performance) in the ICRL loop is a meaningful contribution to the agentic optimization methodology.
Generated Apr 22, 2026
Comparison History (76)
Paper 2 introduces a groundbreaking intersection of LLM agents, formal verification, and compiler design to automate peak-performance GPU kernel generation. While Paper 1 offers a highly practical systems solution for distributed RL, Paper 2 fundamentally shifts how hardware-specific optimizations are developed, replacing rare human expertise with automated, verifiable agentic planning. By achieving state-of-the-art assembly-level throughput and bridging multiple disciplines (AI, programming languages, systems), Paper 2 promises broader and more foundational scientific impact.
Paper 2 demonstrates higher impact due to its extreme timeliness, immediate real-world applicability, and methodological rigor. Automating expert-level GPU kernel optimization addresses a massive bottleneck in AI infrastructure. By uniquely combining LLM agents with formal methods, ARGUS achieves performance matching hand-written assembly on modern hardware (MI300X). While Paper 1 offers a foundational theoretical analysis for a universal ISA, its practical adoption faces severe industry friction and relies on older hardware (T4) for validation, whereas Paper 2 provides a highly actionable solution that instantly accelerates AI workloads.
Paper 2 (ARGUS) presents a more novel and broadly applicable contribution: an agentic framework combining data-flow invariants, formal verification, and LLM-based reasoning for GPU kernel optimization. It addresses a timely intersection of AI agents and systems, achieves near-hand-optimized performance across multiple kernel types (GEMM, attention, MoE) covering 90%+ of LLM inference time, and generalizes to KernelBench. Paper 1 offers strong but narrower engineering optimization specific to MoE communication overlap. ARGUS's methodological innovation (SMT solving, abstract interpretation, in-context RL) and broader impact on automated code generation give it higher potential scientific influence.
Paper 2 presents a highly novel neuro-symbolic approach, combining LLM agents with SMT solvers and data-flow invariants to automate high-performance GPU kernel generation. While Paper 1 offers a valuable systems-level optimization for distributed LLM training, Paper 2 tackles the critical bottleneck of hardware-specific software optimization. By matching hand-optimized assembly performance on complex kernels like Flash Attention, ARGUS bridges compiler theory with generative AI. This has transformative implications for democratizing custom AI hardware deployment and reducing reliance on human experts for low-level kernel optimization.
Paper 1 tackles a fundamental and widely relevant problem—automated GPU kernel optimization—achieving parity with hand-tuned assembly (99-104%) and dramatic speedups (up to 1543x) over prior agentic systems. Its data-flow invariant approach with compiler-verified feedback is genuinely novel and generalizable, impacting LLM inference broadly and bridging AI agents with formal verification. Paper 2 is a solid, timely systems contribution for the emerging dLLM serving niche, but its improvements (8-20%) are incremental and its applicability narrower. Paper 1's stronger novelty, larger gains, and broader cross-field relevance give it higher potential impact.
Paper 1 (ARGUS) appears to offer higher scientific impact due to a more novel, generalizable technical contribution: compile-time data-flow invariants with abstract interpretation/SMT to provide dense diagnostic feedback for GPU-kernel optimization, enabling near hand-tuned performance on major LLM-critical kernels. This advances both programming systems and ML-for-code/agentic optimization, with broad applicability to compilers, DSLs, verification, and heterogeneous computing. Paper 2 (SPORK) is timely and practical, but is a comparatively incremental systems/controller optimization for agentic inference latency with narrower methodological depth and cross-field reach.
Paper 1 addresses a highly timely and critical problem: automatically generating GPU kernels for LLM workloads that match hand-optimized performance. Its results are striking (99-104% of SOTA, orders of magnitude over prior agents) and its novel data-flow invariant approach with verifiable feedback is methodologically innovative and broadly applicable to the fast-growing agentic-coding and ML-systems intersection. Paper 2 tackles important hardware architecture exploration but is more specialized, simulation-based, and less immediately deployable. Paper 1's combination of concrete impact, novelty, and relevance to the LLM inference bottleneck gives it broader potential reach.
Paper 2 (ARGUS) has higher potential impact due to a more novel, generalizable methodology: compile-time data-flow invariants with SMT/abstract-interpretation feedback tightly coupled to an agentic optimization loop. It targets a broad class of GPU kernels (GEMM, attention, MoE) and demonstrates near hand-tuned performance plus strong generalization, implying wide applicability across compilers, DSLs, and AI-for-systems. Paper 1 is strong and timely for MoE serving, but is more domain-specific (serving architecture) and likely narrower in cross-field reuse.
ARGUS demonstrates a fundamentally novel approach to GPU kernel optimization through data-flow invariants and agentic synthesis, achieving near-optimal performance (99-104% of hand-optimized assembly) across critical LLM workloads. Its contributions span compiler design, formal verification (abstract interpretation + SMT), reinforcement learning, and GPU programming—broad cross-field impact. The 2-1543x speedup over existing agentic systems is transformative. KernelFlume, while valuable for elastic LLM serving, addresses a more incremental systems engineering problem (disaggregated attention scaling) with narrower scope. ARGUS's DSL and invariant framework could reshape how GPU kernels are developed across many domains.
ARGUS presents a fundamentally novel framework combining data-flow invariants, abstract interpretation, SMT solving, and reinforcement learning to enable LLM agents to generate near-optimal GPU kernels (99-104% of hand-optimized assembly). This addresses a critical bottleneck in AI systems development with broad implications across compiler design, AI-assisted programming, and GPU computing. The dramatic speedups (2-1543x over existing agentic systems) and near-expert-level performance represent a significant technical breakthrough. Paper 1, while practically valuable for grid management, is more incremental in its contribution, combining existing techniques (workload scheduling, carbon-aware computing) in a relatively small-scale deployment.