PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
Can Hankendi, Rana Shahout, Minlan Yu, Ayse K. Coskun
Abstract
Large language model (LLM) inference has become a dominant workload in modern data centers, driving significant GPU utilization and energy consumption. While prior systems optimize throughput and latency by batching, scheduling, and parallelism, they largely treat GPU power as a static constraint rather than a controllable resource. In this paper, we present a power-aware runtime for LLM serving, PALS, that treats GPU power caps as a first-class control knob and jointly optimizes them with software parameters such as batch size. The system combines lightweight offline power-performance models with a feedback-driven controller to select configurations that satisfy throughput targets while maximizing energy efficiency. We implement PALS within an existing LLM serving framework, vLLM, demonstrating that it requires no model retraining or API changes. Across multi-GPU systems and both dense and mixture-of-experts (MoE) models, PALS improves energy efficiency by up to 26.3%, reduces QoS violations by 4x to 7x under power constraints, and tracks dynamic power budgets. These results highlight the potential of integrating power control directly into LLM inference runtimes, enabling energy-proportional and grid-interactive AI systems.
AI Impact Assessments
(1 model)Scientific Impact Assessment: PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
1. Core Contribution
PALS introduces a power-aware runtime for LLM inference that elevates GPU power caps from static constraints to first-class, dynamically controllable knobs. The key insight is that jointly optimizing hardware-level power limits and software-level parameters (batch size, parallelism) expands the achievable Pareto frontier of efficiency-performance trade-offs beyond what either dimension alone can achieve. The system implements a closed-loop controller combining offline power-performance models (random forests) with PID feedback correction, operating at 500ms granularity within vLLM without requiring model retraining or API changes.
The contribution is particularly relevant for Mixture-of-Experts (MoE) models, where dynamic token routing creates variable compute-communication ratios that fundamentally alter the power-performance relationship. The paper shows that communication-bound MoE models (Qwen-MoE, OLMoE) actually *lose* efficiency beyond ~200W power caps because additional power accelerates communication overhead rather than useful computation—a counterintuitive finding that undermines the default practice of running GPUs at maximum power.
2. Methodological Rigor
The empirical methodology is generally sound. The paper systematically profiles three key dimensions (power caps, batch sizes, parallelism configurations) across eight models spanning dense and MoE architectures. The compute-communication decomposition analysis (Figure 2) provides clear mechanistic explanations for observed behaviors.
However, several methodological concerns arise:
3. Potential Impact
The practical impact potential is significant:
4. Timeliness & Relevance
The paper addresses a genuine and growing need. LLM inference energy consumption is a first-order concern for cloud providers, with major companies publicly committing to sustainability targets while simultaneously scaling AI deployments. The focus on MoE models is well-timed, as the industry is shifting toward sparse architectures (Mixtral, DeepSeek, etc.) for efficiency at scale. The observation that MoE models create unique power-management challenges due to communication overhead is timely and underexplored.
The connection to carbon-aware computing and demand response is forward-looking but not yet validated with real grid signals or carbon intensity data—this remains aspirational rather than demonstrated.
5. Strengths & Limitations
Key Strengths:
Notable Limitations:
Minor Issues:
Overall Assessment
PALS makes a solid systems contribution by demonstrating that joint hardware-software optimization for LLM inference power management is both feasible and beneficial. The MoE-specific insights regarding communication-bound behavior under power scaling are genuinely novel and practically important. However, the evaluation could be more rigorous (statistical reporting, broader hardware platforms, longer-duration experiments), and some claims slightly overstate generality given the experimental scope. The work is a meaningful step toward energy-proportional AI serving, though it falls short of being a comprehensive solution.
Generated May 21, 2026
Comparison History (25)
Paper 2 has higher likely impact due to strong real-world applicability (data-center energy and QoS), clear methodological rigor (implemented in vLLM, evaluated across dense+MoE and multi-GPU), and timely relevance as power becomes a limiting factor for LLM deployment. Its approach (treating power caps as a controllable runtime knob with feedback control) can generalize to many serving stacks and aligns with industry needs (energy proportionality, grid interaction). Paper 1 is novel and valuable for safety evaluation, but its impact depends on adoption of specific datasets/taxonomies and is narrower operationally.
Paper 2 addresses a critical, highly timely issue: the massive energy consumption and environmental impact of LLM inference in data centers. By treating GPU power caps as a controllable resource, it offers immediate, practical benefits for energy efficiency and operational costs across all LLM deployments. While Paper 1 introduces a valuable benchmark for operations research, Paper 2's potential economic and environmental impact gives it a broader and more significant scientific and real-world footprint.
Paper 1 addresses the fundamental and widely-studied problem of object hallucination in vision-language models with a novel training-free approach that achieves state-of-the-art results. Hallucination mitigation is a critical bottleneck for deploying LVLMs reliably, giving it broad impact across multimodal AI. Paper 2 addresses an important but more incremental systems-level optimization (power-aware LLM serving), which, while practically useful, represents engineering refinement rather than a conceptual advance. Paper 1's methodological novelty (region-aware attention recalibration) and its relevance to model trustworthiness give it broader scientific influence.
Paper 2 addresses a more fundamental and broadly impactful question about multi-agent RL training dynamics for LLM workflows, providing novel theoretical insights (gradient dynamics, failure patterns) and a systematic empirical framework across multiple dimensions. Its findings on policy-sharing tradeoffs and failure mechanisms contribute foundational knowledge applicable across many LLM applications. Paper 1, while practically valuable for energy-efficient inference, is more incremental—applying power-capping techniques to LLM serving—with a narrower scope of impact primarily in systems/infrastructure rather than advancing core AI methodology.
AMEL identifies a fundamental and previously undercharacterized bias in LLM-as-judge paradigms, which is now a widespread practice across NLP research, content moderation, and code review. The finding that conversational history systematically shifts LLM judgments has broad implications for any pipeline using LLMs as evaluators, affecting reproducibility across many fields. The large-scale empirical rigor (75K+ API calls, 11 models, 4 providers) and actionable mitigation advice give it high practical relevance. PALS addresses an important but more niche systems optimization problem (energy-efficient LLM serving) with incremental engineering contributions over existing power-capping and scheduling work.
Paper 2 (PALS) likely has higher scientific impact due to strong real-world applicability and timeliness: data-center energy use is a major, immediate constraint, and a runtime integrated into vLLM without retraining lowers adoption barriers. The methodology is concrete (models + feedback control) with clear, measurable gains across hardware and MoE/dense models, suggesting broad systems impact. Paper 1 (LCGuard) is novel and important for emerging latent multi-agent communication safety, but relies on an operational leakage definition (reconstruction) that may not capture all privacy risks and may see slower near-term deployment.
Paper 2 likely has higher scientific impact: it identifies a counterintuitive inverse-scaling failure mode in LLM forecasting under superlinear growth and tail-regime risk, introduces a new benchmark, and shows replication across simulated and multiple real-world domains. The findings affect evaluation methodology (tail-inclusive scoring) and practical deployment in high-stakes settings (finance/epidemiology), with broad relevance to ML safety, calibration, and benchmarking. Paper 1 is a solid systems contribution with clear datacenter applicability, but its novelty and cross-field reach are narrower and more incremental compared to the conceptual and evaluative implications of Paper 2.
Paper 1 likely has higher scientific impact due to a concrete, novel systems contribution (treating GPU power caps as a control variable jointly optimized with batching) with demonstrated, reproducible gains (up to 26.3% energy efficiency, fewer QoS violations) and immediate applicability to widespread LLM serving stacks (integration into vLLM, no retraining). It is timely given datacenter energy constraints and could influence both systems research and production deployments. Paper 2 is important conceptually but is more of a survey/proposal with limited methodological rigor and harder-to-standardize, lower-scalability evaluation evidence.
Paper 1 likely has higher scientific impact due to broad, timely applicability and strong real-world relevance: power/energy-aware LLM serving is a data-center–scale constraint affecting most deployments, and integrating GPU power caps into runtime control (implemented in vLLM without retraining) is a pragmatic, system-level innovation with immediate operational benefits. The methodology (offline models + feedback controller) and evaluation across dense/MoE and multi-GPU settings suggest solid rigor and generality. Paper 2 is impactful for VLM reliability, but may be more benchmark/task- and prompt/framework-dependent and potentially less universally deployable.
Paper 2 addresses a critical, highly timely bottleneck in modern AI: the massive energy consumption and power constraints of serving Large Language Models (LLMs). By dynamically optimizing GPU power caps during MoE model inference without requiring model retraining, it offers immediate, scalable, and high-impact real-world applications for data centers and cloud providers. While Paper 1 introduces a rigorous foundational metric for AI uncertainty, Paper 2's direct solution to the pressing economic and environmental costs of LLM deployment gives it a broader and more immediate potential scientific and industrial impact.