Announcer
The following program features simulated voices generated for educational and technical exploration.
Sam Dietrich
Good evening. I'm Sam Dietrich.
Kara Rousseau
And I'm Kara Rousseau. Welcome to Simulectics Radio.
Kara Rousseau
Tonight we're examining a fundamental inefficiency in modern computing infrastructure: systems consume substantial power even when doing little or no useful work. Traditional computer architecture optimizes for peak performance, but most servers spend significant time at low utilization. The challenge is designing systems where energy consumption scales proportionally with utilization—consuming minimal power when idle and scaling smoothly to maximum power under full load. This is the energy proportionality problem.
Sam Dietrich
From a hardware perspective, this is about component behavior under varying load. A processor at ten percent utilization doesn't consume ten percent of its peak power. It might consume fifty or sixty percent because static power—leakage current, clock distribution, peripheral circuits—doesn't scale with activity. The same applies to memory, network interfaces, storage. Every component has baseline overhead that persists regardless of utilization.
Kara Rousseau
To explore this problem and potential solutions, we're joined by Dr. Luiz André Barroso from Google, whose work on warehouse-scale computing has fundamentally shaped how we think about datacenter efficiency. Dr. Barroso, welcome.
Dr. Luiz André Barroso
Thank you. Delighted to be here.
Sam Dietrich
Let's start with quantifying the problem. What's the typical relationship between utilization and power consumption in modern servers?
Dr. Luiz André Barroso
For typical servers, the relationship is far from linear. A server at zero percent utilization—completely idle but powered on—might consume fifty to sixty percent of its peak power. As you increase utilization, power consumption grows, but the curve is sublinear at the low end and approaches linear only at higher utilizations. This means a server doing very little work is grossly inefficient in energy per useful operation. The problem compounds in datacenters where average utilization is often only ten to thirty percent.
Kara Rousseau
Why is average utilization so low? Is this a provisioning problem or inherent to workload characteristics?
Dr. Luiz André Barroso
It's both. Systems are provisioned for peak load plus headroom for growth and failure handling. But actual load varies significantly—daily cycles, seasonal patterns, unpredictable spikes. You can't run at ninety-five percent utilization because you'd have no capacity for load increases or equipment failures. Additionally, many workloads have bursty characteristics. A web server might be idle between requests. A batch processing system might wait for data. The result is that much of the time, hardware sits underutilized but consuming significant power.
Sam Dietrich
Let's break down where the idle power goes. What are the major contributors?
Dr. Luiz André Barroso
At the processor level, static power from transistor leakage is substantial, particularly in modern deep-submicron processes. The clock network distributes a clock signal across the entire chip whether or not circuits are active, consuming power. Power delivery circuits, voltage regulators, must maintain voltage even at low current draw and have efficiency losses. Beyond the CPU, DRAM refresh cycles consume power continuously. Network interfaces maintain link state. Storage devices keep platters spinning or flash arrays powered. Cooling systems run based on temperature, not strictly on compute load. All these contribute to baseline power that doesn't scale with utilization.
Kara Rousseau
So the ideal would be components that can transition to very low power states when not needed and wake up quickly when work arrives. What prevents this?
Dr. Luiz André Barroso
Transition latency and performance impact. Modern processors have multiple power states—C-states for CPUs, P-states for performance scaling. Deeper sleep states save more power but take longer to wake from. If wake latency exceeds acceptable response time, you can't use deep sleep. Additionally, there's energy overhead for the transition itself—powering down and back up isn't free. For very short idle periods, the energy cost of transitioning can exceed the energy saved. This creates a challenge in workloads with frequent, unpredictable activity.
Sam Dietrich
What about consolidation? If servers are underutilized, why not pack more work onto fewer machines and power off the rest?
Dr. Luiz André Barroso
Consolidation helps significantly. Virtualization and containerization make it easier to move workloads between physical hosts. You can concentrate load onto a subset of machines and put others into deep sleep or turn them off entirely. But there are limits. Some workloads have affinity requirements—they need specific hardware, locality to data, or network topology. Over-consolidation increases risk—if a heavily-loaded machine fails, you've lost capacity for many workloads. And there's overhead in migration and rebalancing. But directionally, consolidation is a powerful tool for improving energy proportionality at the datacenter level.
Kara Rousseau
This suggests the solution isn't purely hardware—software workload management plays a crucial role. How do you coordinate power management across thousands of machines?
Dr. Luiz André Barroso
You need cluster-level orchestration that understands both workload characteristics and power states. The scheduler needs to make placement decisions that consider energy efficiency—preferring to pack work onto already-active machines rather than waking sleeping ones, but avoiding over-concentration that eliminates headroom. You want to create opportunities for machines to enter low-power states by draining work from them deliberately. This requires predicting workload patterns, understanding transition costs, and balancing energy efficiency against performance and reliability. It's optimization under uncertainty.
Sam Dietrich
What about heterogeneity? Could you use different classes of machines optimized for different power-performance points?
Dr. Luiz André Barroso
Absolutely. This is increasingly common. You might have high-performance servers for latency-sensitive workloads and more energy-efficient servers for throughput-oriented batch processing. Some work uses specialized accelerators that have better energy efficiency for specific operations. The challenge is managing this heterogeneity—the scheduler needs to understand which workloads should run on which machine classes, and you need enough volume to justify maintaining multiple server designs. But when done well, heterogeneity improves overall fleet efficiency.
Kara Rousseau
Let's discuss memory. DRAM is a significant power consumer and has poor energy proportionality because of refresh requirements. Are there alternative memory technologies that scale better?
Dr. Luiz André Barroso
This is an active area of development. Technologies like persistent memory—NVDIMM, Intel Optane—don't require refresh and can maintain state without continuous power. But they have different performance characteristics and costs. For systems where you can tolerate slightly higher latency, these technologies can reduce idle power. Another approach is memory power management—putting DRAM ranks into self-refresh or power-down states and concentrating active data onto a subset of memory. But this requires OS and application cooperation to manage data placement.
Sam Dietrich
What about the network? Inter-machine communication doesn't scale down easily because you need to maintain link state and handle incoming packets.
Dr. Luiz André Barroso
Network interfaces are particularly challenging for energy proportionality. An idle NIC still consumes significant power maintaining a 10 or 25 or 100 gigabit link. There's been work on energy-efficient Ethernet standards that allow link rate scaling or transition to low-power modes during idle periods. But in practice, datacenter networks often keep links at full rate to minimize latency and avoid transition overhead. One approach is network-aware workload placement—keeping communicating tasks on the same machine or rack to reduce network power. But this conflicts with other placement goals.
Kara Rousseau
This raises the question of whether energy proportionality should be optimized at the component level, the server level, or the datacenter level. Where does the biggest impact lie?
Dr. Luiz André Barroso
You need all three. At the component level, hardware that transitions efficiently to low-power states provides the building blocks. At the server level, coordinating power states across CPU, memory, storage, and network maximizes efficiency for that machine. At the datacenter level, workload orchestration determines which machines are active and creates opportunities for power savings. The datacenter level often has the biggest impact because you can make coarse-grained decisions—powering down entire servers—that dwarf the savings from fine-grained component-level optimization. But you need the component-level capability to realize those savings.
Sam Dietrich
What about cooling? Does improved energy proportionality in compute equipment translate to cooling savings?
Dr. Luiz André Barroso
Yes, significantly. Cooling power is often proportional to heat dissipation. If you reduce server power consumption by consolidating workloads, you reduce the cooling load. But there's complexity—cooling systems have their own minimum power consumption and control loops. A partially-utilized datacenter might not see linear cooling savings. Additionally, concentrated heat loads from highly-utilized servers can create hotspots that require more aggressive cooling. But overall, reducing server power translates to reduced cooling power, often with a multiplier effect.
Kara Rousseau
Let's talk about measurement and metrics. How do you evaluate energy proportionality in practice?
Dr. Luiz André Barroso
We use several metrics. One is the power consumption curve across utilization levels—plotting power versus CPU utilization. An ideal energy-proportional system would show a straight line from zero to peak. Real systems show a curve with high idle power. Another metric is power utilization effectiveness—the ratio of useful work to total power. You can also measure at the workload level—energy per transaction or per batch job. The challenge is that utilization isn't one-dimensional. A server might have low CPU utilization but high memory or network utilization. You need multi-dimensional metrics.
Sam Dietrich
What progress has been made? Are modern servers more energy-proportional than earlier generations?
Dr. Luiz André Barroso
There's been substantial improvement. Modern processors have more sophisticated power states and faster transition times. Dynamic voltage and frequency scaling is more aggressive. Memory power management has improved. We've reduced idle power as a fraction of peak power. But we haven't achieved true energy proportionality—a machine at near-zero utilization still consumes a significant fraction of peak power. The physics of semiconductor leakage and the need to keep components ready for work impose fundamental limits.
Kara Rousseau
This suggests there might be an architectural mismatch. We're trying to optimize hardware designed for peak performance to also be efficient at low utilization. Should we instead design systems explicitly for variable load?
Dr. Luiz André Barroso
That's an interesting direction. You could imagine processors with asymmetric cores—high-performance cores for peak load and extremely energy-efficient cores for background tasks. Or modular designs where you can power down entire sections. Some mobile processors already do this. The challenge in servers is the workload diversity—you're not just running one application with predictable phases. You have many independent workloads with different characteristics. But there's room for architectural innovation beyond just better power states.
Sam Dietrich
What about edge computing? Does distributing computation closer to users change the energy proportionality equation?
Dr. Luiz André Barroso
Edge computing shifts the problem. Edge locations often have smaller deployments—tens of servers instead of tens of thousands. At this scale, provisioning for peak means lower average utilization. You have less opportunity for statistical multiplexing of load. But edge locations can potentially put more equipment to sleep during predictable low-traffic periods. There's also the possibility of computation migration—moving work between edge and central datacenters based on load and energy availability. But this introduces latency and bandwidth costs.
Kara Rousseau
Let's discuss the interaction with renewable energy. If you have variable power availability from solar or wind, does this change how you think about energy proportionality?
Dr. Luiz André Barroso
Absolutely. With renewable energy, your marginal cost of electricity varies with generation. During peak solar production, power is cheap and abundant. At night, it's more expensive. This creates incentive for temporal load shifting—deferring non-urgent work to periods of high renewable generation. But this requires workload flexibility and storage capacity for buffering work. Energy proportionality becomes even more valuable because you want to minimize baseline power consumption during low-generation periods and be able to ramp up efficiently when cheap power is available.
Sam Dietrich
What about the environmental impact? How significant is the energy proportionality problem from a total energy consumption perspective?
Dr. Luiz André Barroso
Datacenters globally consume several percent of total electricity generation, and this is growing. Poor energy proportionality means we're wasting a significant fraction of that consumption on idle power. If we could improve the idle-to-peak ratio from fifty percent to twenty percent across the fleet, that's a massive energy saving. It's also economically significant—power is a major operational cost. From an environmental perspective, this is one of the highest-leverage optimizations we can make.
Kara Rousseau
Looking forward, what's the path toward better energy proportionality?
Dr. Luiz André Barroso
I see several directions. At the hardware level, continued improvement in low-power states and transition efficiency. More sophisticated power management that can handle multi-dimensional workload characteristics. At the software level, better workload orchestration that creates opportunities for power savings. Increased use of consolidation and migration. At the system level, acceptance that energy proportionality might require different architectures—heterogeneous servers, modular designs, aggressive power state management. And at the datacenter level, integration with renewable energy and grid conditions to optimize temporal load distribution.
Sam Dietrich
But we're unlikely to achieve perfect energy proportionality—a system that consumes near-zero power when idle.
Dr. Luiz André Barroso
Correct. The physics of keeping systems ready for work imposes baseline costs. But we can get significantly better than current systems. Reducing idle power from fifty percent of peak to twenty or thirty percent is achievable with current technology. Beyond that requires more fundamental architectural changes. The goal isn't perfection but continuous improvement toward an ideal that balances energy efficiency with performance, reliability, and cost.
Kara Rousseau
Dr. Barroso, thank you for this thorough examination of energy proportionality and datacenter efficiency.
Dr. Luiz André Barroso
Thank you both. This has been a pleasure.
Sam Dietrich
That's our program for tonight. Until tomorrow, may your utilization match your consumption.
Kara Rousseau
And your idle power approach zero. Good night.