Domain-Specific Data Synthesis for LLMs via Minimal Sufficient Representation Learning

Paper 1 addresses a critical bottleneck in LLM development: domain-specific data scarcity. Its shift from a deductive, prompt-heavy paradigm to an inductive, representation-learning approach is highly innovative. By providing both theoretical proofs for data diversity and strong empirical gains, it establishes a foundational method applicable across countless domains. While Paper 2 presents a practical and novel agentic approach to ASR, Paper 1's contribution to data synthesis will likely have a broader, cross-disciplinary impact on the fundamental training pipelines of large language models.

Paper 2 (DOMINO) likely has higher impact due to broader applicability and clearer real-world utility: scalable domain adaptation and data synthesis from only reference examples, addressing a common bottleneck across many LLM deployments. Its inductive paradigm generalizes beyond any single task and can benefit diverse fields requiring specialized data. It also includes theoretical support (support expansion) and empirical gains on coding benchmarks. Paper 1 is novel and rigorous for multi-LLM cooperative RL, but its impact is narrower (multi-agent reasoning setups) and depends on complex RL training and evaluation within a specific reasoning framework.

Paper 1 (DOMINO) addresses a broadly impactful problem—domain-specific data synthesis for LLMs—with a novel inductive framework combining prompt tuning, contrastive disentanglement, and theoretical guarantees. It has wide practical applicability across many domains and establishes a new paradigm for scalable domain adaptation. Paper 2 contributes a useful evaluation methodology (ADR) for probing LLM reasoning on SAT, but its scope is narrower—primarily benchmarking and evaluation rather than enabling new capabilities. DOMINO's combination of methodological novelty, theoretical grounding, empirical gains, and broad practical utility gives it higher potential impact.

Paper 2 introduces a fundamental insight about LLM aggregation—the 'aggregation paradox'—that challenges the widespread practice of majority voting. Its finding that trace-level synthesis outperforms consensus-based methods across diverse reasoning benchmarks (PhD-level science, competition math, competitive programming) has broader applicability across essentially all LLM deployment scenarios. The provable non-degradation guarantees and the surprising result that a single model with perturbations can outperform heterogeneous model pools represents a paradigm shift in multi-agent reasoning. Paper 1, while solid, addresses a more niche problem (domain-specific data synthesis) with more incremental improvements.

Paper 2 addresses a fundamental bottleneck in LLM development—domain-specific data scarcity—proposing a highly scalable, inductive synthesis framework (DOMINO). Its combination of theoretical proofs and empirical gains on benchmarks suggests broad applicability across multiple AI domains. In contrast, Paper 1 offers a valuable but preliminary application of AI in education, limited by a single-course evaluation and narrower scope, making Paper 2's potential scientific impact significantly higher.

DOMINO introduces a fundamentally new paradigm (inductive vs. deductive) for domain-specific data synthesis, addressing a broadly applicable problem with theoretical guarantees and strong empirical results. Its framework for learning domain representations from examples without explicit descriptions has wide applicability across many domains and tasks. Moment-KV, while solving a real engineering bottleneck in KV cache compression, is more incremental—applying momentum-based temporal aggregation to an existing line of work. Paper 2's broader applicability, theoretical contributions, and paradigm-shifting framing give it higher potential impact.

Paper 1 offers more novel scientific insights by revealing that human perceptual geometry transiently emerges in LLM representations despite no perceptual training, bridging cognitive science and AI interpretability. This finding has broad interdisciplinary implications for understanding both human cognition and LLM internals. Paper 2 presents a solid engineering contribution (DOMINO framework for data synthesis), but is more incremental within the well-explored space of data augmentation and domain adaptation. Paper 1's discovery-driven nature and cross-disciplinary relevance give it higher potential for lasting scientific impact.

Paper 1 addresses a fundamental bottleneck in LLM development—domain-specific data synthesis—proposing an innovative inductive paradigm that does not rely on manual prompt engineering. This methodological advance has broad applicability across numerous fields aiming to adapt LLMs. In contrast, Paper 2 introduces a valuable but highly specialized benchmark for a specific task in materials science, which limits its immediate breadth of impact compared to the foundational LLM methodology proposed in Paper 1.

Paper 2 addresses a critical bottleneck in LLM development (acquiring high-quality domain-specific data) with a scalable, novel inductive framework that eliminates the need for manual prompt engineering. Its approach has broad theoretical and practical applicability across many domains. In contrast, Paper 1, while tackling the important issue of social intelligence, presents a relatively small benchmark (137 items) localized to Chinese contexts, which limits its broader and immediate scientific impact.

Paper 2 presents a novel technical framework (DOMINO) with theoretical guarantees and empirical validation, addressing a concrete and important problem in LLM domain adaptation. It introduces a new inductive paradigm for data synthesis with rigorous methodology including contrastive disentanglement objectives and provable diversity guarantees. Paper 1, while timely, is primarily a conceptual/governance framework introducing terminology (Agentic Technical Debt, Stochastic Tax) without empirical validation or technical depth, limiting its scientific impact to management-oriented discussions rather than advancing the field methodologically.

Paper 2 (DOMINO) introduces a novel framework addressing a fundamental challenge in LLM domain adaptation—synthesizing domain-specific data without explicit domain descriptions. It offers theoretical guarantees, a generalizable inductive paradigm, and demonstrates empirical improvements across coding benchmarks. Its contributions (contrastive disentanglement, minimal sufficient representation learning) are broadly applicable across many domains. Paper 1 is primarily a benchmarking study on a specific dataset (PiSAR) with narrower scope, limited novelty beyond empirical observations about fine-tuning recipe-model mismatches, and less generalizable findings.

Paper 1 has higher potential impact due to a more technically novel, generalizable method (inductive domain definition from examples via minimal sufficient representations, contrastive disentanglement, and prompt tuning) that directly addresses a widespread bottleneck: domain data scarcity for LLM adaptation. It provides theoretical guarantees (support expansion) and measurable performance gains on challenging coding benchmarks, making it broadly applicable across many domains and timely for LLM deployment. Paper 2 is important for responsible evaluation in a specific policy-analytics setting, but its methodological contribution and generality are narrower.

Paper 2 likely has higher impact: it defines a new practical problem setting (Continual Model Routing) aligned with the fast-growing ecosystem of model hubs, and contributes a large-scale benchmark (CMRBench with 2,000+ models) that can standardize evaluation and drive follow-on work. Its proposed method (CARvE) targets scalable, continual updates—highly relevant for real deployments and broadly applicable across tasks/domains. Paper 1 is novel and useful for domain data synthesis, but appears narrower (primarily domain adaptation via synthetic data) and its benchmark impact may be more limited than a widely reusable benchmark + framework for routing in model hubs.

Paper 2 tackles the critical bottleneck of domain-specific data scarcity by introducing a novel inductive synthesis paradigm that operates without explicit natural language prompts. Its approach of learning minimal sufficient representations has broader applicability across various specialized domains than Paper 1's mid-training filtering method. Furthermore, Paper 2 is supported by theoretical proofs and demonstrates strong empirical improvements, suggesting a deeper methodological rigor and higher potential for wide-ranging impact in LLM domain adaptation.

Paper 2 likely has higher scientific impact due to its broadly useful, multilingual diagnostic benchmark (MentalMap) and strong empirical finding (a universal L3 “reasoning cliff”) that reframes debates about whether LLMs build spatial world models from text. The benchmark spans languages, tasks, and diagnostic axes, enabling wide reuse across NLP, cognitive science, and multimodal reasoning research, and includes human baselines that strengthen claims. Paper 1 is novel and applicable for domain adaptation, but its impact is narrower (data synthesis for fine-tuning) and more incremental relative to existing synthesis/prompt-tuning lines.

Paper 1 (DOMINO) addresses a more fundamental and broadly impactful problem—domain-specific data synthesis for LLMs through an inductive paradigm—which has wide applicability across many domains and tasks. It introduces a novel theoretical framework with provable guarantees on distribution coverage, combines prompt tuning with contrastive disentanglement, and establishes a new paradigm shift from deductive to inductive domain specification. Paper 2 (ConMoE) addresses MoE compression, which is important but more incremental and narrower in scope, reformulating existing pruning/merging ideas as prototype reassignment without introducing training or theoretical novelty of comparable depth.

Paper 1 is more novel and broadly impactful: it introduces an inductive paradigm for domain-specific data synthesis from reference examples (no explicit domain description), with a concrete framework (DOMINO) combining minimal sufficient representation learning, contrastive disentanglement, and prompt tuning, plus theoretical support-expansion guarantees and demonstrated gains on implicit-domain coding tasks. This addresses a central bottleneck (domain data acquisition) with clear real-world applicability across many domains. Paper 2 is useful but more incremental (a masking heuristic for SFT stability in low-data regimes) and narrower in scope (mainly SFT-to-RL pipelines on math benchmarks).

Paper 2 (DOMINO) likely has higher impact due to broader applicability and direct performance gains: it addresses a common bottleneck (domain data acquisition) with an inductive, example-driven synthesis paradigm that can generalize across many hard-to-describe domains. It combines a clear methodological contribution (minimal sufficient representation + contrastive disentanglement + prompt tuning), includes theoretical support (distributional support expansion), and shows measurable benchmark improvements. Paper 1 (WIRE) is novel and important for LLM agent safety/policy engineering, but its impact is narrower (prompt-policy conflict diagnostics) and more evaluative than enabling across tasks.