Bridging the Last Mile of Time Series Forecasting with LLM Agents
Yuhua Liao, Zetian Wang, Qiangqiang Nie, Zhenhua Zhang
Abstract
Time series forecasting has advanced rapidly, especially with the emergence of foundation models that show strong zero-shot performance on numerical extrapolation. However, in real-world forecasting settings, a statistically plausible baseline is rarely the final forecast used in practice. Before a forecast becomes decision-ready, it often needs to be revised using weakly structured business context such as holiday effects, campaign plans, external events, historical analogs, and expert feedback. This practical stage remains underexplored in the forecasting literature. In this paper, we formulate this stage as the \textbf{last-mile forecasting} problem and present an LLM-agent framework that sits on top of a forecasting backbone. Our system maintains a unified forecast workspace, invokes tools to retrieve contextual evidence, and converts reasoning trajectories into explicit forecast revision actions under structural safety constraints. It also supports long-horizon forecasting through map-reduce-style decomposition and post-hoc reflection through a memory bank. The resulting system is designed to be controllable and auditable. Through real-world case studies, we show how LLM agents can bridge the gap between statistical prediction and business-ready forecasting.
AI Impact Assessments
(1 models)Scientific Impact Assessment
Core Contribution
The paper introduces and formalizes "last-mile forecasting" — the problem of transforming a statistically generated baseline forecast into a decision-ready forecast through context-aware revisions. The key insight is that in operational settings, raw statistical forecasts are routinely adjusted by human planners using contextual knowledge (holidays, campaigns, expert judgment), and this adjustment process itself can be systematized using LLM agents. The authors propose an action-centric agent framework that maintains a unified forecast workspace, uses tools to gather contextual evidence, applies constrained revision actions (range transforms, point overrides), supports long-horizon decomposition via map-reduce, and accumulates reflection memories across forecasting sessions.
The conceptual framing is the paper's strongest intellectual contribution. By naming and formalizing the "last-mile" as a distinct systems problem — separating numerical extrapolation from contextual revision — the authors carve out a well-motivated niche that bridges judgmental forecasting literature with modern LLM-agent systems. The formulation as constrained sequential revision over an immutable workspace (Equation 4-6) is clean and practical.
Methodological Rigor
This is where the paper has significant weaknesses. The evaluation consists of case studies on a single anonymized time series (daily ticket sales on one air route in China). Three case studies are presented: holiday-aware revision, long-horizon multi-event forecasting, and a self-improvement mechanism over three weekly windows.
While the quantitative results are impressive on paper — 88.2% MAE reduction relative to TimesFM on the Spring Festival window — the experimental design raises concerns:
1. Single dataset, single domain: All experiments use one time series from one domain. There is no evidence the framework generalizes to other domains (retail, energy, finance), other geographies, or series with different characteristics.
2. Baselines are untuned: TimesFM is used without fine-tuning, and Prophet is fitted with holiday information but likely without careful hyperparameter tuning. The comparison is somewhat unfair — the framework leverages detailed historical analog retrieval and calendar context that the baselines don't receive in comparable form.
3. No ablation studies: There are no systematic ablations of the framework components (e.g., what happens without the map-reduce decomposition? Without tool-augmented evidence? With different LLMs?).
4. Self-improvement study is inconclusive: The with-memory configuration actually performs *worse* than no-memory on W2 (MAPE 13.15% vs 12.35%), and only shows improvement on W3. With only three windows and no statistical significance testing, this is suggestive but far from conclusive.
5. The LLM backbone is unspecified: The paper never states which LLM is used, making reproducibility difficult.
6. Anonymized data: While understandable for industry data, this prevents independent reproduction of results.
Potential Impact
The practical framing is compelling and addresses a genuine gap. In industry forecasting workflows, human judgment adjustment is ubiquitous but poorly systematized. The idea of replacing ad-hoc human adjustments with auditable, constrained LLM-agent revisions has real potential for:
The workspace abstraction and audit trail design (immutable baseline, revision trace) are practically valuable and could influence how forecasting systems are built in industry. The concept of separating "forecast generation" from "forecast revision" as distinct system concerns is architecturally sound.
However, the impact is limited by the lack of generalizability evidence and the absence of a benchmark. The authors acknowledge this limitation and suggest building last-mile forecasting benchmarks as future work — but without such a benchmark, community adoption will be slow.
Timeliness & Relevance
The paper is timely. LLM agents are rapidly being deployed across domains, and time series forecasting is a natural application. The positioning between foundation model outputs and operational decision-making addresses a real bottleneck. The related work coverage appropriately spans judgmental forecasting, time series foundation models, and LLM agents for time series.
The paper also arrives at a moment when there's growing skepticism about whether foundation models alone can solve all forecasting problems (citing Ma et al., 2026), making the "post-baseline revision" framing particularly resonant.
Strengths
1. Well-motivated problem formulation: The "last-mile" concept is intuitive, well-articulated, and fills a genuine gap between statistical forecasting and operational use.
2. Clean system design: The workspace abstraction, constrained action interface, and audit trail are well-designed engineering contributions. The immutability of historical observations and baseline forecasts is a smart safety constraint.
3. Detailed revision traces: The appendix provides full revision records, making the system's reasoning transparent and the contribution concrete.
4. Map-reduce decomposition: The approach to long-horizon forecasting through event-window decomposition is practical and elegant.
5. Bridges two literatures: Successfully connects judgmental forecasting (established OR/management science field) with modern LLM agent design.
Limitations
1. Extremely narrow empirical evaluation: One dataset, one domain, no statistical significance testing. The paper reads more as a system description with illustrative examples than as a rigorous empirical study.
2. Circular reasoning risk: The framework performs best precisely on holiday windows where it retrieves historical analogs from the same series — essentially performing pattern matching that a well-configured seasonal model could also do. It's unclear how much the LLM reasoning adds beyond sophisticated lookup.
3. No comparison with simpler context-integration methods: What about Prophet with better holiday specification? Or simple same-period-last-year adjustments? The framework's value over simpler approaches is not established.
4. Scalability unclear: How does this perform when hundreds or thousands of series need revision? The per-series agent interaction could be prohibitively expensive.
5. Missing user studies: Given the emphasis on auditability and controllability, the absence of any human evaluation is a notable gap.
Overall Assessment
This paper makes a valuable conceptual contribution by formalizing last-mile forecasting and proposing a well-designed agent framework. However, it is fundamentally a position/systems paper with illustrative case studies rather than a rigorous empirical contribution. The single-dataset evaluation, unspecified LLM, and lack of ablations or baselines significantly weaken the empirical claims. The ideas deserve development into a full empirical study with diverse datasets, proper baselines, ablations, and ideally user studies.
Generated Jun 2, 2026
Comparison History (20)
Paper 1 addresses the critical challenge of LLM bias mitigation by introducing a novel adaptation of GRPO for online reinforcement learning. Its methodological improvements to AI alignment, along with the release of a compute-efficient reward model, offer broader theoretical and practical contributions to the foundational AI community compared to Paper 2's applied business forecasting framework.
Paper 1 addresses a highly pervasive and underexplored challenge—integrating unstructured real-world context with statistical forecasting. By formalizing 'last-mile forecasting' and proposing an LLM-agent framework, it introduces a novel paradigm with massive cross-industry applicability. While Paper 2 offers a rigorous, domain-specific benchmark for financial reasoning, Paper 1's framework bridges numerical foundation models and LLM reasoning, likely inspiring broader methodological advancements and real-world adoption across multiple scientific and industrial domains.
Paper 1 offers a fundamental theoretical advancement in causal inference by simplifying do-calculus rules and improving estimators. This mathematical rigor and foundational contribution will likely have a deep, enduring impact across multiple scientific disciplines. In contrast, Paper 2 presents a practical, engineering-focused application of LLMs for business forecasting, which, while highly relevant and timely for industry, has narrower long-term scientific implications compared to foundational theoretical breakthroughs.
BigFinanceBench introduces a large-scale, expert-authored benchmark with 928 items and 36,241 rubric points that evaluates the full derivation workflow of financial-research agents, filling a clear gap in AI evaluation methodology. Benchmarks historically drive field-wide progress (e.g., ImageNet, GLUE). The paper provides rigorous evaluation of 10 frontier models with quantified headroom, enabling systematic research. Paper 2 presents an interesting LLM-agent framework for 'last-mile forecasting' but relies on case studies rather than systematic evaluation, limiting its methodological rigor and reproducibility.
Paper 1 has higher potential scientific impact because it defines and operationalizes an underexplored but pervasive real-world problem—“last-mile” forecast revision—bridging statistical forecasting with decision-making context via auditable, tool-using LLM agents and safety constraints. This framing and system concept could generalize beyond forecasting to other domains requiring contextual post-processing of model outputs (planning, analytics, operations). Paper 2 is timely and practical with clear cost savings, but is more incremental (prompt rewriting/translation middleware) and narrower in scientific scope, with impact concentrated in LLM tooling rather than opening a new research problem area.
Paper 1 addresses a critical and highly timely challenge in foundational AI research: improving reinforcement learning (GRPO) for LLM reasoning. By solving sparse reward issues without privileged teachers, it offers significant methodological innovation with broad implications across AI development. Paper 2, while highly practical for business forecasting, proposes an applied agent framework rather than a foundational algorithmic advancement, limiting its overarching scientific impact compared to Paper 1.
Paper 2 (VikingMem) is likely to have higher scientific impact because it proposes a general data-management paradigm (“Memory Base”) plus an implemented system with extensive benchmarked evaluations and latency considerations, addressing a broad, timely bottleneck for stateful LLM applications across many domains. Its contributions are reusable infrastructure and abstractions (events/entities, temporal compression, time-weighted recall) that can influence multiple fields (IR, databases, agents, HCI). Paper 1 is novel and practically motivated but is more domain-specific (forecasting) and relies on case studies, which may limit generalizability and methodological rigor.
Paper 2 introduces a novel conceptual framing ('last-mile forecasting') that addresses a widely recognized but underexplored gap between statistical forecasting and practical decision-making. This has broad applicability across industries (supply chain, finance, retail) and bridges two major research communities (time series forecasting and LLM agents). Paper 1, while thorough, is a benchmark paper for a relatively narrow domain (travel planning). Paper 2's framework—combining foundation models with LLM-agent reasoning, tool use, and structured revision—offers more generalizable methodological contributions and broader cross-field impact.
Paper 2 addresses a broadly applicable problem—bridging statistical forecasting with real-world decision-making using LLM agents—that spans many industries and aligns with the rapidly growing interest in LLM-based agentic systems. Its formulation of 'last-mile forecasting' introduces a novel conceptual framework with wide applicability. Paper 1, while technically sound, targets a narrower domain (autonomous satellite cluster management) with a more incremental architectural contribution (differential transformer for multi-agent RL). Paper 2's timeliness, breadth of potential impact across business forecasting domains, and connection to the LLM revolution give it higher estimated scientific impact.
Paper 1 likely has higher scientific impact because it introduces a broadly reusable benchmark (OR-Space) that enables standardized, lifecycle-faithful evaluation of LLM agents on industrial optimization workflows. Benchmarks often catalyze community progress, support rigorous comparison, and expose failure modes across many methods. Its workspace + multi-stage (build/revise/explain) design is novel and timely for agent reliability, with applications across operations research, software/solver integration, and agent evaluation. Paper 2 proposes a practical agent system for “last-mile forecasting,” but its evidence appears more case-study driven and may be less generalizable without a benchmark-level contribution.
Paper 1 addresses a ubiquitous real-world challenge: incorporating unstructured business context into statistical forecasts. By formalizing 'last-mile forecasting' and leveraging LLM agents for contextual adjustments, it bridges a major gap between academic time-series models and enterprise needs. This high cross-industry applicability and novel integration of agentic workflows with numerical extrapolation gives it a broader potential scientific and practical impact compared to the narrower conversational profiling focus of Paper 2.
Paper 1 introduces a new problem framing (“last-mile forecasting”) and an agentic, tool-using, auditable workflow to integrate weakly structured business context into time-series forecasts—addressing a widely encountered but under-studied real-world gap. Its applications span many industries (retail, supply chain, finance) and could influence both forecasting practice and human-in-the-loop AI system design. Paper 2 is methodologically solid and timely, but mainly offers an efficiency/router improvement within multi-hop QA pipelines, a narrower incremental advance with more limited cross-domain impact.
Paper 1 addresses a fundamental scientific question about LLM reasoning capabilities by exposing critical statistical and methodological flaws in a prominent benchmark. Its emphasis on rigorous evaluation methods and discovery of confounding variables provides a crucial course correction for AI evaluation. While Paper 2 offers a valuable applied framework for real-world forecasting, Paper 1 has broader scientific implications for how the AI research community measures and understands the core cognitive capacities of foundation models.
Paper 2 has higher potential impact due to a more novel methodological contribution: a self-supervised RL framework that removes the need for mental-state annotations while distilling model-based Theory-of-Mind reasoning into efficient single-pass inference. This targets a central, broadly relevant capability for interactive agents (HCI, robotics, assistive AI, social cognition modeling) and is timely given growing focus on agentic systems. Paper 1 is practically valuable, but appears more like a systems/application framing atop existing forecasting backbones with case-study validation, likely yielding narrower methodological and cross-field influence.
Paper 1 addresses a broadly applicable problem—integrating contextual business knowledge into time series forecasting via LLM agents—that affects numerous industries and bridges a significant gap between statistical models and practical decision-making. Its framework introduces novel concepts (last-mile forecasting, forecast workspaces, safety constraints, memory banks) with wide applicability. Paper 2 solves a narrower problem in semiconductor inspection with a relatively simple technique (input binarization), showing moderate improvements on one dataset. While useful, its impact is limited to a specific niche, whereas Paper 1's framework could influence forecasting practice across many domains.
Paper 1 introduces a novel conceptual framework ('last-mile forecasting') and an innovative LLM-agent architecture to bridge a critical gap between statistical predictions and real-world business applications. While Paper 2 provides a rigorous and valuable analysis of an existing technique (persona prompting), Paper 1's integration of LLM reasoning, tool use, and structured constraints to solve practical quantitative forecasting problems offers higher potential for transformative impact across both AI research and enterprise applications.
Paper 1 introduces a large-scale benchmark and a novel execution-based evaluation protocol for a highly complex and emerging frontier: LLM-driven 3D world synthesis. Its methodological rigor (2,026 tasks, runtime state probing) and broad applicability across AR/VR, simulation, and spatial computing give it higher potential scientific impact compared to Paper 2, which focuses on a more specialized, business-oriented application of LLM agents in time series adjustment.
Paper 2 is more likely to have higher scientific impact because it proposes a concrete, timely framework (LLM agents + tools + safety constraints) for an under-studied but ubiquitous real-world gap in forecasting: incorporating weakly structured context to revise predictions. Its approach is broadly applicable across industries and domains that rely on forecasts, and aligns with active research on agentic LLM systems, auditability, and controllable decision support. Paper 1 is largely a perspective/call-to-action; while important and potentially high-impact long-term, it is less methodologically rigorous and less immediately actionable.
AgentPLM introduces a more technically novel framework combining reasoning-augmented decoding with a new training objective (CAPO) for protein design, addressing fundamental limitations of protein language models. It demonstrates state-of-the-art results across multiple rigorous benchmarks (enzyme design, antibody optimization, thermostability, PPI design). The protein design domain has enormous biomedical impact potential. Paper 2 addresses a practical but more incremental problem (last-mile forecasting adjustments) with case studies rather than systematic benchmarks, and its contributions are more engineering-oriented than scientifically foundational.
Paper 1 is likely higher impact due to a clearer, broadly useful benchmark contribution with measurable, reproducible evaluation: an interactive, long-horizon EHR simulation that exposes process–outcome gaps and provides ontology-grounded scoring. This directly enables rigorous comparison of agentic/LLM clinical decision-making and information acquisition, a timely area with high stakes and cross-cutting relevance to interactive evaluation, sequential decision-making, and safety. Paper 2 frames an important applied problem, but relies on case studies and a system architecture whose methodological rigor and generalizable evaluation signal are less clear, making its scientific contribution potentially less durable.