Computation, Not Memory: Chain-of-Thought and State Tracking in LLM Agents
Pedram Mouseli, Massieh Moayedi
Carefully controlled, honestly scoped study that cleanly separates generation from retention of CoT in an agent, but the toy environment, two same-family models and partly expected findings limit its reach.
Abstract
Language-model agents often reason before they act, and the value of monitoring that reasoning depends on how it relates to their actions. We ask how an agent's actions depend on its reasoning when the correct action depends on a changing state. In a small file-system task, background events announce which file's lock has been toggled but not its new value. The agent therefore has to reconstruct the current locks from its history before it can move files safely. With Qwen3-8B and Qwen3.5-9B, we separate two roles of reasoning: generating it before each action, and retaining earlier reasoning in the prompt. Most of the reduction in lock-violating commands comes from reasoning written at the moment of action. Retaining earlier reasoning adds little to rule-following, although it helps one model complete the task. The benefit depends on what the reasoning says. Filler text, reasoning written for another history and an accurate table of the state do not reproduce it. When we give the agent reasoning written for a history that differs in one lock value, the agent acts on that reasoning even against its own history. Without reasoning, linear probes recover the state from the model's activations about as well as a text classifier recovers it from the transcript. However, decoding accuracy does not predict which interventions change the agent's actions. In this setting, the benefit of reasoning for rule-following comes mainly from generating it at each step. The state that can be decoded without reasoning is a limited guide to how the agent acts.
AI Impact Assessments
(1 model)Core Contribution
The paper asks what role chain-of-thought (CoT) plays for an LLM agent that must track a changing state. It uses a small file-system task in which background events toggle file locks without announcing the new value, so the agent must infer the current lock state from its history. It separates two roles of reasoning: *generating* reasoning at the moment of action, and *retaining* earlier reasoning in the prompt (CoT as computation versus CoT as memory). The main findings, on Qwen3-8B and Qwen3.5-9B:
The significance is mainly conceptual: it separates computation from memory in agent CoT, and it argues that decodable state is a weak guide to what drives action.
Methodological Rigor
The design is careful for a small-scale study.
Weaknesses:
Potential Impact
The findings bear on CoT monitorability, agent memory and context management, and probing-versus-causal interpretability.
Impact is limited by the toy environment (three files, eight turns), two models from one family, and a single task type. Most individual findings are in line with expectations, and the generation-versus-retention result is plausible given the Qwen chat templates. The influence is therefore likely moderate. It is most useful as a clean experimental template and as a data point for the CoT-monitoring and mechanistic-interpretability communities.
Timeliness
The paper is highly timely. CoT monitorability (Korbak et al.; Baker et al.), the faithfulness debate, and agent memory management are all active areas. The paper offers a concrete empirical decomposition with an inexpensive, replicable setup (single GPU, about 34 GPU-hours total). The conclusion that decodability is a poor guide to causal use is a relevant caution for activation-based monitoring proposals.
Strengths
Limitations
Overall
This is a solid, careful, modest-scope empirical study. It is likely to be cited in discussions of CoT monitorability and agent context design, but unlikely to be a field-defining contribution.
Generated Oct 20, 2026
Comparison History (0)
Carefully controlled, honestly scoped study that cleanly separates generation from retention of CoT in an agent, but the toy environment, two same-family models and partly expected findings limit its reach.