Self-Distillation Policy Optimization via Visual Feedback: Bridging Code and Visual Artifacts
Haoyu Dong
Abstract
Code-generating large language models (LLMs) increasingly produce visual artifacts such as charts, web pages, and slides by writing programs that are executed by non-differentiable renderers, committing to code before observing the render. As a result, otherwise executable code often yields artifacts with visually salient defects, including overlapping elements, clipped text, broken alignment, low contrast, and overflow. We study visual-feedback self-distillation for code-generated visual artifacts. We propose Visual-SDPO, a self-distillation policy-optimization framework that treats rendered visual feedback as privileged context for a weight-sharing teacher and distills this feedback into a coding student. To make supervision spatially targeted rather than uniform, we introduce Visual-Grounded Code Credit Weighting, which traces each detected defect back to the code statements responsible for the affected elements and amplifies the distillation signal on those statements. A sequence-level GRPO (Group Relative Policy Optimization) term complements the dense token-level objective by rewarding executable, visually high-quality rollouts, while failed executions remain learnable through the self-distillation path by passing execution errors as privileged context to the teacher. We instantiate Visual-SDPO for chart, web/UI, and slide generation with a unified Qwen3-VL-8B-Instruct backbone. Across chart-to-code, UI-to-code, and slide-generation benchmarks (ChartMimic, Design2Code, and AeSlides), Visual-SDPO improves over the zero-shot base by more than 10 absolute points in the primary metric and over GRPO by at least 2.4 points, with fewer training steps and no added inference-time cost.
AI Impact Assessments
(1 models)Scientific Impact Assessment: Visual-SDPO
1. Core Contribution
Visual-SDPO addresses a genuine and increasingly important gap in code-generating LLMs: the disconnect between the code that models produce and the visual artifacts those programs render. The core idea is elegant — use the rendered visual output as "privileged context" for a weight-sharing teacher model during self-distillation, so that visual understanding transfers to a student that operates without rendering at inference time. The paper introduces two key innovations:
1. Visual-Feedback Self-Distillation: A teacher that shares weights with the student but additionally conditions on the rendered artifact (or structured rubric thereof), with KL divergence driving token-level learning.
2. Visual-Grounded Code Credit Weighting: A mechanism that traces visual defects back to responsible code statements via region-to-code mapping (through runtime introspection or VLM-based attribution), then amplifies the distillation gradient on those statements proportionally to IoU overlap with defect regions.
The combination with sequence-level GRPO creates a complementary training signal — dense and localizable from distillation, holistic and execution-grounded from RL.
2. Methodological Rigor
The paper demonstrates reasonable rigor with a clear progressive ablation design across three domains (charts, web/UI, slides). Each domain uses the same Qwen3-VL-8B-Instruct backbone, providing controlled comparisons. The ablation chain (zero-shot → SFT → OPSD → GRPO → Visual-SDPO) effectively isolates each component's contribution.
However, several concerns limit confidence:
3. Potential Impact
The paper addresses a practical problem with growing relevance. As LLMs are increasingly deployed for generating charts, dashboards, web UIs, and presentations, the visual quality gap is a real bottleneck. Key impact vectors include:
4. Timeliness & Relevance
This paper is highly timely. The intersection of code generation and visual artifact production is an active frontier, as evidenced by the many concurrent works cited (most from 2025-2026). The paper positions itself well against:
The self-distillation approach occupies a useful middle ground. The paper also capitalizes on the maturity of VLMs (Qwen3-VL) that can serve as both policy and visual feedback processor.
5. Strengths & Limitations
Strengths:
Limitations:
Overall Assessment
Visual-SDPO presents a conceptually appealing framework that addresses a real problem at the intersection of code generation and visual quality. The idea of privileged visual self-distillation with spatially-grounded credit assignment is novel and well-motivated. The empirical results across three domains are encouraging, though the absence of variance estimates, isolated credit-weighting ablations, and multi-scale experiments weakens the empirical case. The 10+ point improvements over zero-shot baselines are impressive, though the more meaningful 2.4+ point gains over GRPO need statistical validation. The work makes a solid conceptual contribution that could influence how the community thinks about training code-generating models for visual outputs.
Generated Jun 10, 2026
Comparison History (17)
Visual-SDPO introduces a novel self-distillation framework bridging code generation and visual rendering with spatially-targeted credit assignment, addressing a broadly relevant problem as LLMs increasingly generate visual artifacts. It demonstrates strong empirical gains across multiple benchmarks with practical benefits (no inference cost increase). While TouchThinker makes solid contributions to tactile reasoning with a large-scale dataset, its impact is constrained to a narrower community (tactile/embodied AI). Visual-SDPO's methodology is more generalizable, timely given the explosion of code-generating LLMs, and addresses a higher-demand application space.
Paper 1 addresses a fundamental challenge in general-purpose AI agents: long-horizon reasoning and context window limitations. Its hierarchical memory mechanism and RL-based retrieval offer a scalable solution applicable across diverse agentic tasks. While Paper 2 presents an innovative self-distillation method for visual code generation, its scope is primarily limited to multi-modal UI and chart generation. Paper 1's foundational approach to agent working memory gives it a broader potential scientific impact across the broader AI community.
Paper 2 (Visual-SDPO) presents a more concrete and technically novel contribution with clear, measurable improvements across multiple benchmarks. It addresses a well-defined, growing problem (visual defects in code-generated artifacts) with a principled method combining self-distillation, visual feedback, and spatially-targeted credit assignment. The approach is broadly applicable across chart, web/UI, and slide generation, with strong quantitative results. Paper 1 addresses an important but more abstract problem in AI-driven scientific discovery, but its evaluation on 40 tasks with subjective quality judgments is less rigorous, and the framework contributions are more incremental in the agent/LLM discovery space.
Paper 1 introduces a highly novel methodological framework bridging code generation and visual rendering through multi-modal self-distillation. Its approach has broad, cross-disciplinary applications across data science, web development, and design. While Paper 2 offers valuable domain-specific contributions to pulmonary medicine, Paper 1 presents a more broadly applicable and timely advancement in foundational multi-modal LLM alignment and agentic workflows.
Paper 2 likely has higher impact: it introduces a new training framework (Visual-SDPO) combining privileged visual feedback, credit assignment from defects to code, and GRPO, showing consistent multi-domain gains on established benchmarks with no inference-time cost—high novelty plus clear, broad applications in UI/chart/slide generation and code agents. Paper 1 provides an important benchmark for control-intervention awareness in LLM safety, but is primarily evaluative; impact depends on downstream adoption and may be narrower and more scenario-specific than a general optimization method.
Paper 2 introduces a novel training methodology bridging code generation and visual feedback, addressing a widespread issue in LLMs. Its framework has broad, immediate real-world applications across web development, data visualization, and UI design. Paper 1 offers a useful benchmark, but its focus on prediction market traces is narrower in scope compared to the generalizable and highly relevant multimodal LLM advancements presented in Paper 2.
Paper 2 has higher potential impact due to a broader and timely application space (code-to-visual artifacts for charts, UI, and slides), addressing a widely observed non-differentiable rendering bottleneck. Its visual-feedback self-distillation and defect-to-code credit assignment are novel and can generalize to many program-to-environment settings. Reported gains are large across multiple benchmarks with no inference-time cost. Paper 1 offers a valuable, efficient fix to credit misassignment in tool-augmented RL, but the contribution is more specialized to multimodal search agents and specific tool-call structures.
Paper 2 proposes a highly novel methodological framework combining visual feedback with policy optimization for code generation, addressing the challenging non-differentiable rendering problem. Its application across multiple domains (charts, web/UI, slides) and integration of reinforcement learning (GRPO) with multimodal LLMs suggest broader methodological impact and real-world applicability compared to Paper 1, which primarily introduces a new benchmark.
Paper 2 likely has higher scientific impact due to stronger timeliness and cross-field relevance: monitoring and auditing multi-agent systems targets a growing deployment paradigm and connects to AI safety, governance, and security. The “budget-aware, continual” oversight framing and active inspection actions provide a generally applicable methodology with clear real-world applications (enterprise agents, decision support, high-stakes domains). Paper 1 is technically novel and useful for code-to-visual generation quality, but its impact is narrower (visual artifact rendering) and more application-specific, whereas Paper 2’s ideas can generalize across many multi-agent settings.
Visual-SDPO introduces a novel self-distillation framework bridging code generation and visual rendering—a broadly applicable problem as LLMs increasingly generate visual artifacts. It combines multiple technical innovations (visual-grounded credit assignment, privileged teacher distillation, GRPO integration) with strong empirical results across three diverse benchmarks using a unified model. Paper 2 addresses a narrower niche (compliance/audit rule refinement) with a practical but less generalizable contribution. While Paper 2 has real deployment validation, Paper 1's methodological novelty, broader applicability across code-to-visual tasks, and potential to influence both LLM training and visual generation research give it higher scientific impact.