Toward AI VIS Co-Scientists: A General and End-to-End Agent Harness for Solving Complex Data Visualization Tasks
Haichao Miao, Zhimin Li, Kuangshi Ai, Kaiyuan Tang, Chaoli Wang, Peer-Timo Bremer, Shusen Liu
Abstract
The ability to inspect, interpret, and communicate complex data is crucial for virtually any scientific endeavor, but often requires significant expertise outside the core domain ranging from data management and analysis to visualization design and implementation. We present an end-to-end agentic harness that, based on only the data and a high level description of the tasks, independently designs custom visual analysis applications (VIS apps). This represents an important step towards a general AI co-scientist envisioned by many as an autonomous system that can autonomously execute long horizon tasks based on high-level directions. Our proposed VIS co-scientist is an essential component of this broader AI co-scientist vision: a harness that can autonomously analyze data and design visualization solutions using a collection of agents and specialized skills that coordinate exploratory analysis, plan, configure the environment, implement, validate the interface, and most importantly evaluate the overall task completion. Each stage produces document and instruction artifacts that guide downstream work and enable iterative refinement. We validate this approach on IEEE SciVis Contests spanning multiple science and engineering fields. These contests serve as ideal proving grounds because they encode real-world complexity: ambiguous requirements, diverse data modalities, design trade-offs, and task-driven validation. Given only the data and target tasks, our system autonomously produces functional single-page VIS Apps with verified linked-view behavior, highly customized to domain experts' specified tasks and needs.
AI Impact Assessments
(1 model)Scientific Impact Assessment
Core Contribution
This paper introduces the concept of a "VIS co-scientist" — a multi-agent AI system that autonomously generates complete, interactive visualization applications (VIS Apps) from raw data and high-level task descriptions. The system orchestrates multiple specialized subagents (Exploratory Data Analyzer, Planner, Environment Builder, VIS Designer, Evaluator) through a structured harness that produces intermediate artifacts, enabling iterative refinement and traceability. The key novelty is the end-to-end pipeline: from data profiling through environment setup, visualization design, browser-based validation, to task-completion evaluation — all without human intervention beyond the initial prompt.
The system is validated on IEEE SciVis Contests (2021–2026), which serve as realistic benchmarks involving ambiguous requirements, heterogeneous data formats, and domain-specific analytical tasks.
Methodological Rigor
The methodology has both strengths and notable gaps. The multi-agent architecture is well-motivated, with clear role separation and artifact-based communication that improves transparency. The use of Playwright-MCP for browser-based validation is a practical choice that enables mechanical verification of linked-view behavior, console errors, and rendering correctness.
However, the evaluation is thin for a systems paper claiming significant advances:
1. Limited quantitative evaluation: The structured expert audit involves only 5 author-team members (introducing potential bias), using Likert-scale assessments on a single case study (2025 contest). The other contest years receive only qualitative descriptions ("mechanically valid").
2. No ablation study: The contribution of individual components (EDA agent, Planner, Evaluator feedback loops) is never isolated. The memory system is described but explicitly not evaluated — the authors state results "do not yet depend on prior memory retrieval."
3. Baseline comparison is weak: The comparison is against a "baseline coding agent" (same model without the harness), which predictably produces standalone plots rather than coordinated apps. There is no comparison against other agentic frameworks, human-produced solutions at equivalent effort levels, or even systematic comparison against contest winners.
4. Reproducibility concerns: The code is described as "currently waiting upon code review completion" for open-source release, and the system depends on proprietary models (GPT-5.4 via OpenAI Codex), making independent verification difficult.
Potential Impact
The paper addresses a genuine need: visualization application development is a bottleneck in scientific workflows, requiring expertise in data management, visual encoding design, frontend development, and domain understanding. An autonomous system that produces functional, interactive VIS Apps could democratize data exploration across scientific domains.
The multi-agent harness design pattern — with explicit artifact handoffs, layered validation, and specialized subagents — could influence how agentic systems are designed for other complex, multi-stage creative tasks beyond visualization. The use of SciVis Contests as benchmarks is clever and could establish a standard evaluation paradigm for visualization agents.
However, the practical impact may be limited by several factors: the generated apps lack visual design novelty (acknowledged by the authors), the system struggles with 3D visualization and temporal reasoning, and the token costs (5.5M+ tokens for one run) are substantial. The expert audit scores for visual encoding quality (mean 3.8) and domain insight extraction (as low as 2.4 for some statements) suggest the system is not yet reliable enough for unsupervised scientific use.
Timeliness & Relevance
The paper is highly timely, situated at the intersection of two active trends: AI co-scientist systems (Google's AI co-scientist, AI Scientist v2, etc.) and LLM-based visualization agents. The argument that visualization is an essential but underexplored component of the AI co-scientist vision is well-made and fills a conceptual gap in the literature. The positioning against SciVisAgentBench (from some of the same authors) suggests awareness of the evaluation infrastructure needed.
Strengths
Limitations
Additional Observations
The paper reads more as a short/workshop paper (which it appears to be, given its length and venue positioning) than a full research contribution. Several design choices reference very recent (2026) publications and tools, suggesting rapid development. The plan to submit a VIS co-scientist entry to the 2026 SciVis Contest is an excellent future validation strategy. The conceptual framing of what a VIS co-scientist should be (Figure 2) provides useful scaffolding for the community, even if the current instantiation is preliminary.
Generated May 22, 2026
Comparison History (23)
While Paper 2 offers critical, life-saving advancements in ICU ventilator decision support, Paper 1 presents a broader scientific paradigm shift. By introducing an autonomous AI co-scientist for complex data visualization, Paper 1's methodology can accelerate discovery and data interpretation across virtually all scientific and engineering disciplines. Its end-to-end agentic harness addresses a universal bottleneck in research, giving it a higher potential for widespread, cross-disciplinary scientific impact.
Paper 1 presents a broadly applicable end-to-end AI agent system for autonomous data visualization—a capability relevant across virtually all scientific domains. Its validation on real IEEE SciVis Contest datasets demonstrates practical utility for domain scientists. Paper 2 addresses an important but narrower technical problem (semantic recoverability for tool agents during failures). While rigorous, its impact is more confined to runtime/systems engineering for LLM agents. Paper 1's breadth of applicability, alignment with the high-profile AI co-scientist vision, and cross-domain relevance give it higher potential impact.
Paper 2 addresses the broader and more timely challenge of AI co-scientists for data visualization, with wider cross-disciplinary applicability. Its end-to-end agentic system for autonomously generating visualization applications from raw data has immediate practical utility across virtually all scientific domains. Paper 1, while rigorous in formalizing AI governance compliance, addresses a narrower infrastructure concern with limited evaluation scope (24 validation runs, one scenario). Paper 2's validation on real IEEE SciVis Contest datasets demonstrates broader generalizability and aligns with the high-impact trend of autonomous AI agents.
Paper 2 addresses the broader and more impactful vision of AI co-scientists for data visualization, with clear real-world applications across multiple scientific domains. It presents an end-to-end system validated on real-world IEEE SciVis Contest benchmarks spanning diverse fields, giving it wider interdisciplinary impact. While Paper 1 makes a solid contribution to LLM multi-agent planning with a well-defined problem (epistemic miscalibration), its scope is narrower and more incremental (9.75% improvement). Paper 2's potential to democratize complex data visualization across sciences gives it greater breadth of impact.
Paper 1 tackles foundational challenges in formal mathematics and program verification, offering mathematically rigorous guarantees for code correctness. While Paper 2 presents a highly useful tool for automating data visualization, Paper 1's findings fundamentally advance AI's logical reasoning capabilities, expose critical flaws in current verification benchmarks, and establish compiler-in-the-loop agentic systems as state-of-the-art. This has profound, far-reaching implications for software security, AI safety, and the future development of reliable autonomous systems.
SkillOpt introduces a fundamentally novel paradigm—treating agent skills as optimizable external state analogous to weight-space optimization—with rigorous methodology (52 evaluation cells, 7 models, 3 harnesses, ablations, transfer experiments). Its broad applicability across any LLM agent system, strong empirical gains (+19-25 points), and transferability across models and environments suggest wide adoption potential. Paper 1, while impressive as an engineering contribution to visualization, is more domain-specific and represents incremental progress in applying agentic AI to a narrower application area rather than introducing a general-purpose methodology.
Paper 2 has higher potential impact: it proposes an end-to-end agentic system that autonomously builds functional visualization applications from data and high-level tasks, validated on real-world IEEE SciVis contests across multiple domains. This has clear, broad applications (scientific analysis, communication, tool-building automation) and is timely given rapid advances in agentic LLM/VLM systems. Paper 1 is a valuable diagnostic/evaluation framework for VLM spatial-number grounding, but its impact is more specialized and primarily benchmarking/analysis rather than enabling a new general capability with immediate cross-field utility.
Paper 2 has higher potential impact due to its broader applicability and stronger real-world utility: an end-to-end agent harness that autonomously builds task-specific visualization apps can benefit many scientific domains and accelerate data-driven discovery. Its validation on IEEE SciVis contests suggests practical relevance and methodological rigor around complex, ambiguous, multi-modal tasks. Paper 1 is timely and novel in EI evaluation with real multi-turn, user-annotated conversations, but its impact is more specialized to LLM assessment and conversational safety/alignment rather than a cross-disciplinary workflow enabler.
Paper 1 presents a novel, concrete end-to-end agentic system for autonomous data visualization that is validated on real-world benchmarks (IEEE SciVis Contests), demonstrating tangible outputs. It addresses a broadly applicable problem—automating complex visualization tasks—with methodological rigor and clear innovation in multi-agent coordination. Paper 2, while comprehensive, is primarily a review/survey chapter discussing how AI could enhance serious games, offering less novelty and no new system or empirical validation. Paper 1's concrete contribution and broader cross-domain applicability give it higher impact potential.
Paper 1 introduces an AI co-scientist framework for autonomous data visualization, addressing a fundamental bottleneck in data analysis across all scientific domains. By automating complex visual analysis and validating on real-world IEEE SciVis contests, it demonstrates broad applicability and high potential to accelerate multidisciplinary research. In contrast, Paper 2 presents a useful software engineering framework for LLM tool deployment, which greatly improves developer efficiency but has limited direct impact on fundamental scientific discovery methodologies. Therefore, Paper 1 exhibits significantly higher scientific innovation and broader potential impact.