Data centers are where the Internet and cloud services live, and so they have been getting lots of public attention in recent years. If we read technology news or research papers, it’s not uncommon that we see IT giants, like Google and Facebook, publicly discuss and share the designs of mega-scale data centers they operate. But, another important type of data center –– multi-tenant data center, or commonly called “colocation”/”colo” –– has been largely hidden from the public and rarely discussed (at least in research papers), although it’s very common in practice and located almost everywhere, from Silicon Valley to the gambling capital, Las Vegas.
Unlike a Google-type data center where the operator manages both IT equipment and the facility, multi-tenant data center is a shared facility where multiple tenants house their own servers in shared space and the data center operator is mainly responsible for facility support (like power, cooling, and space). Although the boundary is blurring, multi-tenant data centers can be generally classified as either a wholesale data center or a retail data center: wholesale data centers (like Digital Realty) primarily serve large tenants, each having a power demand of 500kW or more, while retail data centers (like Equinix) mostly target tenants with smaller demands.
The mathematics of planet earth is a joint initiative from a consortium of mathematical sciences organizations around the world (organized nominally by DIMACS) that has the goal of showcasing how mathematics can be useful in tackling our world’s problems. It started last year as a year-long focus, but has now expanded and will continue for the coming years as well. I’ve been to a few events organized under this program, but the reason for this post is to highlight the recent workshop on “Data-aware energy use” organized at UCSD a week or so ago.
Throughout our work on data center workload management over the past few years, one common theme that emerged was that of online convex optimization. Whether we were looking at load shifting, dynamic right-sizing, geographical load balancing, or data center demand response, we consistently ended up with an online convex optimization problem that we needed to solve. So, I wanted to do a short post here highlighting something particularly surprising (at least to us) that we uncovered last year.
A bit of news on the data center front, for those who may have missed it: Facebook recently announced the deployment of a new power-efficient load balancer called “Autoscale.” Here’s their blog post about it.
Basically, the quick and dirty summary of the design is to adapt the number of active servers so that it’s proportional to the workload, and adjust the load balancing to focus on keeping servers “busy enough” so that they don’t end up in a situation where lots of servers are very lightly loaded.
So, the ideas are very related to what’s been going on in academia over the last few years. Some of the ideas are likely inspired by Anshul Ghandi and Mor Harchol-Balter et al.’s work (who have been chatting with Facebook over the past few years), and it’s actually quite similar in the architecture to the “Net Zero Data Center Architecture” developed by HP (that incorporated some of our work, e.g. these papers, which are joint with Minghong Lin, who now works with the infrastructure team at Facebook).
While this isn’t the first tech company to release something like this, it’s always nice to see it happen. And, it will give me more ammo to use when chatting with people about the feasibility of this sort of design. It is amazing to me that I still get comments from folks about how “data center operators don’t care about energy”… So, to counter that view, here’re some highlights from the post:
“Improving energy efficiency and reducing environmental impact as we scale is a top priority for our data center teams.”
“during low-workload hours, especially around midnight, overall CPU utilization is not as efficient as we’d like. […] If the overall workload is low (like at around midnight), the load balancer will use only a subset of servers. Other servers can be left running idle or be used for batch-processing workloads.”
Anyway, congrats to Facebook for taking the plunge. I hope that I hear about many other companies doing the same in the coming years!
The typical story surrounding data centers and energy is an extremely negative one: Data centers are energy hogs. This message is pervasive in the media, and it certainly rings true. However, we have come a long way in the last decade, and though we certainly still need to “get our house in order” by improving things further, the most advanced data centers are quite energy-efficient at this point. (Note that we’ve done a lot of work in this area at Caltech and, thanks to HP, we are certainly glad to see it moving into industry deployments.)
But, the view of data centers as energy hogs is too simplistic. Yes, they use a lot of energy, but energy usage is not a bad thing in and of itself. In the case of data centers, energy usage typically leads to energy savings. In particular, moving things to the cloud is most often a big win in terms of energy usage…
More importantly, though, the goal of this post is to highlight that, in fact, data centers can be a huge benefit in terms of integrating renewable energy into the grid, and thus play a crucial role in improving the sustainability of our energy landscape.
In particular, in my mind, a powerful alternative view is that data centers are batteries. That is, a key consequence of energy efficiency improvements in data centers is that their electricity demands are very flexible. They can shed 10%, 20%, even 30% of their electricity usage in as little as 10 minutes by doing things such as precooling, adjusting the temperature, demand shifting, quality degradation, geographical load balancing, etc. These techniques have all been tested at this point in industry data centers, and can be done with almost no performance impact for interactive workloads!
Two years ago, Greenpeace put out a report titled “How clean is your cloud,” taking many of the IT giants to task for their lack of commitment to sustainability in their data centers. Now, a few years later, Greenpeace is still at it and has been pushing hard with a mixture of yearly public praise/shaming (or maybe they’d prefer the term “public education”) about the commitment and progress companies are making toward a sustainable cloud.
When reading the most recent report “Clicking clean,” it is really quite amazing how far the industry has come. While there is still room for improvement, even the companies Greenpeace critiques are light-years ahead of where the industry was five years ago. Apple, which was the black sheep of the initial report, has now committed to 100% renewable energy for its cloud, while Amazon, which was ahead of the curve in the initial report, is really hit hard.
James Hamilton has had two interesting recent posts about local renewable generation for data centers that are definitely worth a read for folks interested in “sustainable data centers”: Solar at Scale and Datacenter Renewable Power Done Right.
Sustainable data centers
It’s always interesting to hear perspectives on “sustainable data centers” from industry, because there is a big diversity still in how companies are moving to make their data centers sustainable. Some companies are going with a “local” approach, where renewable generation (in a variety of forms) is integrated on-site, while others are going with a more global approach, where renewables are placed somewhere else on the grid (often nearby, but not always). An example of the former is Apple and an example of the latter is Google.