Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective
Junpeng Zhang, Lei Cheng, Guoxi Zhang, Hua Cai, Qing Xu, Quanshi Zhang
Abstract
This paper explores a scientific question in supervised fine-tuning (SFT): why SFT is broadly effective for small-scale deep neural networks, yet can produce inconsistent or even detrimental effects when applied to large language models (LLMs). Recent advances in interaction-based explanations suggest that interactions between words/tokens provide a faithful metric for quantifying the inference patterns encoded by LLMs. We find that the evolution of interactions during SFT can effectively explain the inconsistent effectiveness of SFT for LLMs. Specifically, we find that (1) SFT primarily removes noise-like interactions, while rarely acquiring reliable new interactions. (2) This denoising stage is extremely brief, after which continued fine-tuning tends to introduce overfitted interactions. We validate these findings across multiple LLMs and datasets. Our findings provide new insights into early stopping and offer practical guidance for LLM training.
AI Impact Assessments
(1 model)Scientific Impact Assessment
Core Contribution
This paper investigates why supervised fine-tuning (SFT) yields inconsistent results for LLMs—sometimes improving and sometimes degrading performance—using the framework of AND-OR interactions between input tokens as a diagnostic lens. The key findings are: (1) SFT primarily removes noise-like interactions (high-order, mutually canceling, non-generalizable patterns) rather than learning new reliable ones; (2) this denoising phase is extremely brief (often within ~1000 training steps), after which continued training introduces overfitted interactions. The paper frames the SFT process as having two distinct phases—a short beneficial denoising stage followed by a prolonged harmful overfitting stage—which reconciles seemingly contradictory views in the literature about SFT's effectiveness.
Methodological Rigor
The paper builds on the AND-OR interaction framework of Chen et al. (2024), which has theoretical guarantees (universal matching property) and empirical validation of sparsity. The methodology for categorizing interactions into removed, preserved, and newly emerged types is clean and well-defined. The authors validate findings across seven model-dataset combinations (Qwen2.5-3B/7B, Llama-2-7B, Llama-3-8B, Gemma-3-4B across GoEmotions, Unilaw-R1-Data, and Databricks-Dolly-15k), providing reasonable breadth.
However, there are notable methodological concerns:
1. Input variable selection: The analysis operates on a small number of sampled words (~10 per input), which significantly reduces the input space. While this is justified for computational tractability, it raises questions about whether the observed interaction dynamics truly capture the full picture of SFT's effects on the model.
2. LoRA-only fine-tuning: All experiments use LoRA, a parameter-efficient method that updates a low-rank subspace. It remains unclear whether the findings generalize to full fine-tuning, which is the more classical SFT setup and arguably the setting where overfitting concerns are most acute.
3. Limited task diversity: The datasets span emotion classification, legal QA, and instruction-following, but the evaluation focuses primarily on token-level prediction scores rather than downstream task metrics. The connection between interaction quality and actual task performance could be more explicitly established.
4. Generalizability metric: Using a single different-architecture LLM as the baseline for measuring interaction generalizability is somewhat brittle. The choice of baseline model could influence the generalizability scores.
Potential Impact
The paper offers several potentially valuable contributions:
However, the practical impact is tempered by the computational cost of interaction extraction (exponential in the worst case, though approximations exist). The paper acknowledges this limitation but doesn't demonstrate a practical early-stopping pipeline that would be usable at scale.
Timeliness & Relevance
The paper addresses a timely question as the field debates the relative merits of SFT versus RLHF and other post-training paradigms. The finding that SFT's primary value is denoising rather than learning new capabilities resonates with recent observations (e.g., LIMA's "less is more" thesis) and the shift toward reinforcement learning-based training (DeepSeek-R1). The interaction-based explanation framework is gaining traction, and applying it to understand training dynamics is a natural and timely direction.
Strengths
1. Clear narrative: The two-phase characterization (denoising then overfitting) is intuitive and well-supported by multiple complementary metrics (order distribution, generalizability, cancellation ratio).
2. Theoretical grounding: The universal matching property provides formal justification for using interactions as faithful representations of model behavior.
3. Comprehensive visualization: The interaction distribution plots across training steps effectively communicate the temporal dynamics.
4. Multiple validation perspectives: Section 3.3 validates that preserved interactions form the inference backbone from three independent angles (cancellation, individual contribution, sufficiency for prediction).
5. Breadth of models: Testing across four model families strengthens the generality claims.
Limitations & Weaknesses
1. Causal claims are overstated: The paper describes correlations between interaction quality metrics and SFT stages but doesn't establish causation. The claim that "SFT primarily removes noise" is an interpretation of the interaction dynamics, not a mechanistic explanation.
2. No actionable early-stopping algorithm: Despite claiming practical guidance, the paper doesn't propose or evaluate a concrete early-stopping criterion. When exactly should training stop? How sensitive is this to hyperparameters?
3. Missing downstream evaluation: The analysis focuses on interaction-level metrics without systematically connecting them to actual task performance (accuracy, F1, etc.) across the training trajectory.
4. Scale limitations: The largest model tested is 8B parameters. Whether these findings hold for models at 70B+ scale—where the SFT debate is most contentious—is unknown.
5. Single fine-tuning method: Only LoRA is tested, limiting generalizability to full fine-tuning or other PEFT methods.
6. Computational cost: The exponential cost of exact interaction extraction limits the scalability and reproducibility of the approach, despite available approximations.
Overall Assessment
This paper provides an interesting and well-executed empirical study using interactions to characterize SFT dynamics. The two-phase narrative is compelling and consistent across experiments. However, the work is primarily descriptive rather than prescriptive—it offers explanations but limited actionable tools. The gap between the diagnostic insights and practical utility (e.g., a working early-stopping algorithm with demonstrated performance improvements) reduces the immediate impact. The restriction to LoRA and moderate-scale models also limits the strength of the conclusions for the broader SFT debate.
Generated May 19, 2026
Comparison History (26)
Paper 1 addresses a fundamental and highly timely issue in large language models—understanding the mechanics and inconsistencies of supervised fine-tuning (SFT). Given the widespread adoption of LLMs, providing a faithful metric (token interactions) and practical guidance (early stopping) for SFT offers immense breadth of impact. Paper 2 presents a strong methodological advancement in world models for offline RL, but its impact is relatively confined to the robotics and control domains compared to the ubiquitous relevance of LLM training dynamics.
Paper 1 introduces a novel framework (TTExplore) addressing a significant and underexplored problem—agents operating under implicit rules—with a concrete RL training pipeline and demonstrated improvements of 14-19 points across five tasks. It has broader practical impact for LLM-based agents. Paper 2 provides interesting theoretical insights into SFT dynamics using interaction-based explanations, but its contributions are more analytical/explanatory rather than introducing new methods, limiting its immediate practical impact despite offering useful guidance on early stopping.
Paper 1 addresses a highly pressing real-world problem—the environmental footprint of data centers—by introducing a novel, dynamic Electricity-Computation-Water nexus framework. Its use of differentiable optimization to directly mitigate physical water stress offers significant interdisciplinary impact across AI, energy systems, and environmental sustainability. While Paper 2 provides valuable theoretical insights into LLM fine-tuning, Paper 1's potential to drive immediate, large-scale resource conservation in global computing infrastructure gives it broader societal and scientific significance.
Paper 2 is more likely to have higher scientific impact: it addresses a timely, widely relevant question about supervised fine-tuning behavior in LLMs and proposes an explanatory mechanism (interaction evolution) with empirical validation across multiple models/datasets, yielding actionable guidance (early stopping, avoiding overfitted interactions). This combination of novel insight + methodological evidence + broad applicability to LLM training and alignment suggests impact across ML research and practice. Paper 1 is a useful systems/platform contribution, but its novelty is more engineering-oriented and evaluations appear narrower (internal case studies, preference judgments).
Paper 1 addresses a fundamental scientific question about why SFT behaves differently for LLMs versus small networks, providing mechanistic insights through interaction-based analysis. Its findings on denoising dynamics and overfitting have broad implications for LLM training practices and early stopping strategies. Paper 2, while practically useful, is primarily an engineering/systems contribution presenting an autonomous research platform with limited internal evaluation. Paper 1 offers deeper theoretical understanding with validated findings across multiple models, contributing more lasting scientific knowledge to the field.
Paper 1 likely has higher impact due to a concrete algorithmic contribution (SERL) that improves long-horizon credit assignment for multi-turn LLM agents using environment feedback, with strong empirical gains on widely used agent benchmarks (ALFWorld, WebShop). This is timely for tool-using agents and has clear real-world applicability and potential to generalize across interactive settings. Paper 2 offers valuable interpretability insights into SFT dynamics and practical guidance (early stopping), but is more explanatory/diagnostic and may translate less directly into broadly adopted training methods.
Paper 2 introduces a highly innovative, interdisciplinary approach by formalizing LLM self-correction through control theory. By replacing ad-hoc prompting with a systematic, closed-loop framework and introducing rigorous new metrics (convergence, overshoot), it establishes a strong mathematical foundation for a critical area of LLM research. While Paper 1 provides valuable mechanistic insights into SFT, Paper 2's methodological rigor and potential to create a new paradigm for evaluating and improving LLM reasoning give it a higher potential for broad scientific impact.
Paper 2 addresses LLM hallucinations, a critical bottleneck for real-world deployment. Its training-free, inference-time algorithm (TRACE) demonstrates massive empirical gains across a wide variety of models without requiring labels or fine-tuning. While Paper 1 provides valuable theoretical insights into SFT dynamics, Paper 2 offers an immediate, highly scalable, and universally applicable solution to a more pressing problem, likely leading to broader adoption and higher scientific impact.
Paper 2 likely has higher scientific impact: it tackles a core, timely scientific discrepancy in LLM training (why SFT can fail), proposes a mechanistic interaction-based explanation, and reports validation across multiple models/datasets, offering actionable guidance (early stopping) relevant to broad ML/LLM research and practice. Paper 1 introduces a developer framework to balance SDK usability and vendor neutrality; while useful and applicable, it is more engineering/tooling-focused with narrower scientific novelty and cross-field impact compared to Paper 2’s generalizable insights into learning dynamics.
Paper 2 addresses a fundamental theoretical and practical issue in Supervised Fine-Tuning (SFT) for LLMs. Its insights into how SFT affects token interactions and its guidance on early stopping have broad implications across the entire AI and NLP community. While Paper 1 is an innovative and highly useful application of AI to materials science, Paper 2's contributions impact the core methodologies used to train foundation models, yielding a wider breadth of impact.