OneLA: Scaling Linear-Attention Decoding to Large Beams in Generative Recommendation

Scientific Impact Assessment: OneLA

1. Core Contribution

OneLA is a systems/inference-optimization paper addressing a specific and newly-identified bottleneck: serving linear-attention (Gated DeltaNet) models under large-beam decoding for generative recommendation (GR). The central observation is that in GR, hundreds of candidate beams all originate from an identical prompt-derived recurrent state and diverge only through a handful of short decode-time transitions. The paper turns this structural redundancy into three coupled mechanisms: (1) a compact representation storing one shared post-prefill state (S_ctx) plus append-only "GDN Transition Records" (GTRs) that require O(d_v + d_k) instead of O(d_v·d_k) per beam; (2) a projection-replay trick that never reconstructs the full recurrent state matrix, exploiting the fact that GDN interacts with prior state only through two vector contractions (key/query projections); and (3) a lightweight ancestry index that decouples logical beam evolution (pruning/reordering/fan-out) from immutable physical record placement, plus a fused GPU kernel reusing S_ctx across beams. The reported payoff is 1.54–2.46× end-to-end decode speedup and large reductions in state capacity and DRAM traffic.

2. Methodological Rigor

The evaluation is reasonably thorough for a systems paper. They test against the natively-supported paths of vLLM (FullState, ReplaySSM), SGLang, FlashInfer, and TensorRT-LLM, across 405 operator shapes spanning request counts, beam widths, decode lengths, and nine recurrent geometries. A notable strength is the Controlled FullState baseline, which isolates the OneLA decode pipeline from its state representation and shows it reproduces vLLM FullState within 6.9–9.3% — this properly attributes the gains to the state-management idea rather than incidental engineering. Numerical exactness is verified (max abs error 4.88×10⁻⁴ vs FP32), which is important because the method is a mathematically exact reformulation, not an approximation. The performance breakdown (latency, capacity, DRAM traffic, arithmetic intensity via roofline) convincingly links the compact representation to the observed speedups.

Weaknesses: experiments use a single small model (0.8B Qwen3.5), narrow prompt lengths (1K/5K), and no real serving-throughput/latency-SLA measurement under production traffic. The comparison baselines are general-purpose systems not designed for this workload, so the large operator-level speedups (40×) partly reflect that mismatch rather than a head-to-head with a workload-specialized competitor (e.g., xGR is cited but not benchmarked against).

3. Potential Impact

GR is an industrially significant and fast-growing paradigm (OneRec, OneReason, PLUM, S-GRec are all cited industrial systems), and Kuaishou authorship signals real deployment intent. As frontier models increasingly adopt linear/gated attention, the intersection of linear attention + large-beam decoding is a genuine emerging need. The technique is directly deployable and could be adopted by industrial GR serving stacks. However, the impact scope is inherently narrow: it is tightly bound to GDN-style recurrences and the specific GR workload profile (long shared prompt ≫ short SID suffix, wide dynamic beam). It is unlikely to influence broader ML or even general LLM serving, since conventional single-sequence decoding does not exhibit the same shared-origin/short-suffix structure that makes projection replay cheap.

4. Timeliness & Relevance

Highly timely. It identifies "beam width as a new scaling dimension for recurrent linear attention" — a genuinely fresh framing that sits at the confluence of two active trends (linear attention adoption and generative recommendation). It addresses a concrete bottleneck that existing systems (vLLM/SGLang) handle poorly.

5. Strengths & Limitations

Strengths: clean problem identification; an elegant, mathematically exact reformulation (projection replay is the intellectual highlight); well-designed controlled ablation; broad operator sweep; verified numerical fidelity. Clear writing and effective figures.

Limitations: single-model/single-family scope; no code or public dataset release, hurting reproducibility; no downstream recommendation-quality validation (mitigated by exactness); baselines are not workload-specialized; no production-scale throughput/tail-latency analysis. The contribution is a targeted engineering optimization rather than a conceptual breakthrough — several ingredients (prefix state sharing, append-only records, ancestry indices) echo established ideas from PagedAttention and prefix caching, recombined for the recurrent-state setting.

Additional Observations

The projection-replay insight — recognizing that only two vector projections of the state are ever needed, so the full matrix need never be reconstructed even during replay — is the most transferable idea and could inspire similar treatments in other recurrent-state serving contexts. Reproducibility is moderate: the equations and kernel design are specified well enough for an expert to reimplement, but the industrial workload and code are not released.

Overall, this is a solid, well-executed systems paper with clear industrial relevance in a timely niche, but limited breadth and modest conceptual novelty cap its expected influence to a specific slice of the GR-serving subfield.