Back to Rankings

SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects

Puyi Wang, Yuhao Wang, Linjie Li, Zhengyuan Yang, Kevin Qinghong Lin, Yangguang Li, Yu Cheng

May 19, 2026arXiv:2605.19587v1
cs.AI
Share
Scorecard· 5/16
6.0/10 impact

Abstract

Indoor scene synthesis underpins embodied AI, robotic manipulation, and simulation-based policy evaluation, where a useful scene must specify not only what the environment looks like, but also how its objects are structured. Existing pipelines, however, typically represent generated content as static meshes and inherit articulation only from curated asset libraries, which limits object-level controllability and prevents new interactable assets from being produced on demand. We address this gap by formulating physically interactable indoor scene synthesis as programmatic world generation, and present SceneCode, a framework that compiles a natural language prompt into an executable, code-driven indoor world rather than a collection of opaque meshes. A room-level agentic backbone first turns the prompt into a structured house layout and emits per-object AssetRequests through a planner--designer--critic loop. Each request is then routed to one of five code-generation strategies and converted into a synthesized part-wise Blender Python programs that are validated through an execution-guided repair-and-refine loop. The resulting programs are compiled into simulation-ready assets, and exported as SDF for physics simulation. A persistent scene-state registry links object requests, executable programs, rendered geometry, and simulation assets, turning scene assembly into a traceable and locally editable world-building process. We evaluate SceneCode across scene-level synthesis, object-level asset quality, human judgment, and downstream robot interaction. Results show that executable world programs improve prompt-faithful indoor scene generation and produce assets with cleaner mesh structure, and simulator-loadable articulation metadata. Project page: https://scene-code.github.io/.

AI Impact Assessments

(1 model)

Scientific Impact Assessment: SceneCode

1. Core Contribution

SceneCode reframes indoor scene synthesis as programmatic world generation, where a natural language prompt is compiled into executable Blender Python programs that produce part-decomposed, articulated, simulation-ready 3D assets. The central insight is that code serves as a natural representation for interactable scenes because it can unify geometry, part hierarchy, material assignment, articulation metadata, and simulation packaging in a single editable artifact. This contrasts with prior approaches that either retrieve static meshes from curated libraries (Holodeck, LayoutVLM, HSM) or generate opaque meshes via image-to-3D pipelines (SceneSmith's non-articulated branch).

The key novelty is the five-strategy routing system paired with an execution-guided repair-and-refine loop: each object request is dispatched to one of five specialized VLM-based code-generation strategies (wall art, static furniture, simple/structured manipulands, articulated objects), each with tailored geometric construction priors. This routing addresses the practical reality that a single universal prompt to a VLM produces unreliable geometry across diverse indoor object categories. The persistent scene-state registry that links object requests, programs, rendered geometry, and simulation assets is a useful architectural choice for traceability and local editability.

2. Methodological Rigor

The evaluation is multi-faceted but has notable gaps. The paper evaluates on 30 prompts from SceneEval-100 across six room categories, measuring both scene-level metrics (10 SceneEval metrics) and object-level metrics (6 mesh/material metrics), supplemented by a user study and MuJoCo demonstrations.

Strengths in evaluation design:

  • The paired-difference user study design (Δ = SceneCode − Baseline within each trial) is statistically appropriate, controlling for rater identity and prompt sampling.
  • Object-level comparison against SAM 3D Objects on the same requests with the same reference images is fair.
  • The 95% confidence intervals are reported throughout.
  • Weaknesses:

  • The user study involves only 9 participants split into groups of 3, which is statistically underpowered for drawing robust conclusions. The confidence intervals on ΔPF reflect this (e.g., [-0.05, 0.34] for SceneSmith).
  • The robot interaction evaluation (Section 4.4) is purely qualitative—three MuJoCo demonstrations with no quantitative metrics on joint accuracy, motion range correctness, or task success rates. This is a significant gap given that simulation-ready articulation is a headline claim.
  • No ablation study on the routing mechanism, ObjectPlan verification, or the execution-guided validation loop. The appendix shows one qualitative comparison (Figure 6) of generic vs. route-specific prompts, but no systematic quantitative ablation.
  • SceneCode does not lead on several important metrics (OOR, OAR, SUP, ACC, OPC), and the paper's explanations for these gaps—while reasonable—are not experimentally validated.
  • The computational cost is substantial: average ~$21.73 per scene and ~7.5 hours wall-clock time, with a maximum of ~17 hours. This raises scalability concerns that are acknowledged but not addressed.
  • 3. Potential Impact

    The paper addresses a genuine bottleneck in embodied AI: the dependence on curated articulated asset libraries limits the diversity and customizability of simulation environments. SceneCode's ability to generate novel articulated objects on demand (e.g., a cabinet with a specific number of drawers and glass doors) is valuable for:

  • Robotic manipulation research: Generating diverse training environments with interactable objects without manual asset creation.
  • Embodied AI benchmarking: Producing controllable scene variations for systematic evaluation.
  • Sim-to-real transfer: Creating simulation-ready scenes with physically meaningful articulation.
  • However, the practical impact is tempered by the high computational cost and the acknowledged visual quality gap relative to retrieval/image-to-3D approaches. The primitive-based construction produces clean geometry but may lack the visual fidelity needed for visual policy training.

    4. Timeliness & Relevance

    This work is timely on multiple fronts:

  • The rise of VLM-based code generation (GPT-4, Gemini) makes programmatic 3D asset synthesis increasingly feasible.
  • The embodied AI community's growing need for scalable, diverse simulation environments creates clear demand.
  • The shift toward simulation-ready scene generation (SceneSmith, HSM, ProcTHOR) positions this work within an active research trajectory.
  • The formulation of scenes as executable programs aligns with the broader trend of "code as representation" in AI (Code as Policies, ProgPrompt), extending it to 3D world generation.

    5. Strengths & Limitations

    Key Strengths:

  • Novel and well-motivated formulation: scenes as executable programs with intrinsic articulation.
  • Comprehensive pipeline design with thoughtful engineering (five routing strategies, plan verification, execution-guided repair).
  • Clean separation of concerns between room-level planning and object-level code generation.
  • Demonstrates clear advantages in mesh quality metrics (zero non-manifold edges, ~4.4× fewer UV islands) and semantic fidelity (CNT, ATR).
  • The persistent scene-state registry enables local editability, which is practically valuable.
  • Notable Weaknesses:

  • Underpowered user study (n=3 per group).
  • No quantitative evaluation of articulation quality (joint accuracy, motion range correctness).
  • Missing ablation studies for key design decisions.
  • High computational cost (~$22/scene, ~7.5 hours average) limits practical scalability.
  • Visual realism trails SceneSmith (Δ_Realism = -0.34), acknowledged but not addressed.
  • The 30-prompt evaluation set, while spanning six categories, is relatively small for establishing generalization.
  • Additional Observations

    The extensive appendix (prompt designs, code listings, cost statistics) supports reproducibility, though the reliance on proprietary VLMs (likely GPT-4/Gemini) introduces a dependency on commercial APIs. The paper would benefit from testing with open-source VLMs. The claim of "executable world programs" is compelling conceptually but the execution is more of a well-engineered pipeline than a fundamentally new algorithmic contribution.

    Rating:6.2/ 10
    Significance 6.5Rigor 5.5Novelty 6.5Clarity 7

    Generated May 20, 2026

    Comparison History (27)

    Lostvs. Emotional intelligence in large language models is fragmented across perception, cognition, and interaction

    Paper 2 has higher potential impact due to its broad relevance across AI alignment, HCI, and psychology. While Paper 1 offers a strong technical contribution for embodied AI, Paper 2 addresses a critical, ubiquitous issue in foundational LLMs: the superficiality of RLHF-driven emotional responses ('stochastic empathy'). By bridging psychometric theory with LLM evaluation, Paper 2 provides a foundational benchmark that challenges current scaling assumptions and provides a roadmap for AI safety, making it highly influential for the future deployment of AI in human-facing and clinical applications.

    gemini-3.1-pro-preview·May 26, 2026
    Wonvs. Beyond Inference-Only Deployment: Comparing Weight-Based Consolidation Against Cascading Compaction

    SceneCode addresses a fundamental challenge in embodied AI and robotics simulation—generating physically interactable, editable indoor scenes from language prompts as executable programs rather than static meshes. This has broad impact across embodied AI, robotic manipulation, and simulation-based training. Paper 1, while offering a useful practical contribution on weight-based personalization for LLMs, is narrower in scope (n=10 conversations, single-GPU LoRA fine-tuning) and addresses a deployment optimization rather than opening a new research direction. SceneCode's programmatic world generation paradigm is more novel and impactful across multiple fields.

    claude-opus-4-6·May 26, 2026
    Wonvs. ConceptM$^3$oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology

    Paper 2 introduces a highly innovative approach to scene synthesis by using executable programs to generate articulated, interactable 3D environments. This overcomes a major bottleneck in embodied AI and robotics, which currently rely on static meshes or limited curated libraries. Its broad applicability across robotics, simulation, and spatial computing gives it widespread potential impact, whereas Paper 1 has a narrower, though important, focus on computational pathology.

    gemini-3.1-pro-preview·May 26, 2026
    Lostvs. Can AI Make Conflicts Worse? An Alignment Failure in LLM Deployment Across Conflict Contexts

    Paper 2 addresses a critical gap in AI safety evaluation—LLM alignment in conflict contexts—that has immediate real-world consequences for journalism, humanitarian work, and public discourse in fragile societies. It proposes the first evaluation framework for this domain, which could influence alignment benchmarking standards across the industry. While Paper 1 (SceneCode) is technically impressive for embodied AI and scene synthesis, Paper 2's breadth of societal impact, timeliness given rapid global LLM deployment, and cross-disciplinary relevance (AI safety, conflict studies, policy) give it higher potential scientific and societal impact.

    claude-opus-4-6·May 22, 2026
    Lostvs. Latent-space Attacks for Refusal Evasion in Language Models

    Paper 1 addresses a critical and highly active area of research: LLM safety, alignment, and jailbreaking. By providing a principled understanding of refusal suppression in latent space and demonstrating state-of-the-art attack success across many models, it exposes fundamental vulnerabilities in current AI systems. While Paper 2 offers a valuable framework for embodied AI simulation, the broader implications, urgency, and cross-disciplinary relevance of securing foundational models give Paper 1 a higher potential for immediate and widespread scientific impact.

    gemini-3.1-pro-preview·May 22, 2026
    Lostvs. Generative Recursive Reasoning

    GRAM introduces a fundamental new framework for neural reasoning by making recursive reasoning models probabilistic, enabling multi-trajectory computation with theoretical grounding in variational inference. This addresses core questions about how neural systems should implement extended computation, with broad applicability across reasoning, generation, and constraint satisfaction. While SceneCode is a strong engineering contribution for embodied AI scene synthesis, GRAM's conceptual innovation in combining recursive latent reasoning with generative modeling has broader potential impact across multiple fields of AI research, offering a new paradigm for inference-time scaling and probabilistic reasoning.

    claude-opus-4-6·May 21, 2026
    Lostvs. Generative Recursive Reasoning

    GRAM introduces a fundamental architectural innovation for neural reasoning—turning deterministic recursive reasoning into probabilistic multi-trajectory computation. This addresses a core challenge in AI (how to implement extended computation in neural systems) with broad theoretical and practical implications across reasoning, generation, and inference-time scaling. SceneCode is a strong engineering contribution for indoor scene synthesis but is more application-specific. GRAM's framework-level contribution to reasoning architectures has wider potential impact across multiple fields and research directions.

    claude-opus-4-6·May 21, 2026
    Lostvs. AutoResearchClaw: Self-Reinforcing Autonomous Research with Human-AI Collaboration

    While Paper 1 offers an innovative approach to scene synthesis critical for embodied AI and robotics, Paper 2 targets the automation of scientific discovery itself. By developing an iterative, multi-agent autonomous research system with self-healing execution and human-AI collaboration, Paper 2 has the potential for a massive multiplier effect across all computational disciplines, offering significantly broader scientific impact.

    gemini-3.1-pro-preview·May 20, 2026
    Wonvs. Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption

    SceneCode introduces a novel framework bridging indoor scene synthesis with executable, editable programmatic representations—addressing a concrete gap in embodied AI, robotics, and simulation. Its contributions span multiple fields (scene generation, articulated object synthesis, robot interaction) and offer immediately usable outputs (simulation-ready assets). Paper 1 provides useful empirical insights for multi-model LLM scheduling but is primarily a benchmarking/profiling study offering guidelines rather than a new method or system, limiting its transformative impact.

    claude-opus-4-6·May 20, 2026
    Wonvs. Neurosymbolic Learning for Inference-Time Argumentation

    Paper 2 likely has higher impact due to stronger real-world applicability and broader cross-field relevance: it enables editable, executable scene generation with articulated assets directly usable in simulators (SDF), benefiting robotics, embodied AI, graphics, and simulation. The programmatic representation and execution-guided repair loop are novel and practically enabling, with clear downstream evaluation in robot interaction. Paper 1 is timely and methodologically interesting for trustworthy claim verification, but its impact is more domain-specific (NLP/argumentation) and may face adoption friction due to dataset/task specificity and dependence on argument generation quality.

    gpt-5.2·May 20, 2026