It takes only a few minutes for servers to overheat during a power outage. Yet while statistics show more data centers are backing up their servers with uninterruptible power supply (UPS) technology, too few of them back up their cooling units.
New research from Opengate Data Systems, a Hubbardston, Mass.-based research firm, found that a data center running at 5,000 watts per server cabinet will face an automatic and thermal shutdown in just over three minutes during a power outage. Data centers running with 10,000 watts or more of server equipment will shut down in less than a minute.
"Power and cooling are hand in hand," said Jerry Allen, network operations center manager at Georgia State University in Atlanta. "If you lose power and you lose your cooling, you often don't have generators big enough to handle the cooling pumps and air handlers. You're going to lose your ability to cool your servers."
Active Power Inc., an Austin, Texas-based vendor of power and UPS technology, commissioned the study. The vendor contends these few minutes are critical because many data centers have only their IT hardware connected to uninterruptible power supplies, not cooling units. The cooling technology often relies on generator power or a restoration of utility power. In the seconds that it takes for a generator to start, the HVAC system often powers down. Once the generator starts supplying power, the HVAC system will take several minutes to begin cooling the room again. And in the case of some small and medium-sized businesses (SMBs), data center generators often aren't strong enough to provide sufficient cooling.
Andrew Kutz, a data center strategies analyst at Burton Group Inc. in Midvale, Utah, said he wasn't sure why many data centers lack an emergency cooling solution. He said it could be because so many were designed in the 1970s, when companies had fewer servers that produced far less heat.
"They did not take into consideration the amount of heat that can be produced now," Kutz said. "When they designed them, existing systems were perfectly fine. They could survive without HVAC for a period of minutes. And a lot of servers had self-contained cooling. They still have self-contained cooling, but that's not sufficient now."
Data center chilling solutions
Although solutions are not easy, nor cheap, Kutz said there are ways around the problem.
"You could put your HVAC on UPS and just buy more UPSes," Kutz said. "Some people relocate their data centers to a cold-air environment and expose the data center to the external environment during an outage. It naturally cools it. Or you can try detecting power failures by monitoring. If there is a spike, you can begin to shut down some of your servers so you reduce your overall energy footprint in case of a failure."
Some vendors offer chilled water systems that can cool a data center during power failures, but these technologies tend to be expensive for SMBs.
"To put in a chilled water system just runs too high for us," Allen said. "It's way too expensive and way too much of a capital expenditure."
And there are a growing number of companies offering less expensive (and sometimes unique) alternatives. Active Power, for example, has products for medium-sized companies that can supply emergency cooling to the data center without major changes to its infrastructure. Instead of using batteries to supply emergency power, the company's CoolAir UPS and CoolAir DC units use compressed air to spin power-generating turbines. The turbines produce frigid air as a byproduct, which can be used to cool the data center. Active Power says its research study shows this technology can prevent a thermal shutdown in a data center cool for more than 10 minutes, 30 seconds during a power outage.
Allen has been using a CoolAir UPS for 18 months and is currently installing a second unit. He said his data center has a spotty power supply, so having emergency power and cooling in place is critical. Since he installed his first CoolAir UPS, his data center has suffered about 17 power surges or losses.
"I'm actually occupying a space that was a book repository," Allen said of his data center. "We borrowed it until we had a new data center. That was 20 years ago, and we're still running off the building's power. The power coming into the building is not as clean as it should be. If there's a spike or a drop, the UPS handles that."
When there is an interruption to utility power at Allen's data center, three things happen: His generator kicks in and powers mission critical servers; the CoolAir UPS devices support power on other high-priority servers; and a third group of servers that are noncritical are on unprotected power. When an outage happens, those servers shut down immediately.
Servers running Allen's critical business applications receive power from a continuously spinning flywheel that can produce power for 10 seconds until his generator kicks in. Then the generator powers those critical business applications.
Although the generator also powers Allen's cooling system, he said he isn't convinced it will provide enough juice to keep his data center from overheating.
How do you test that in real life? Shut down the power and wait to see what happens?
Jerry Allen, network operations center manager, Georgia State University
"My chiller and pumps are actually on my 750 kilowatt generator; however, that does not say it would be able to cool the whole data center. I've never had a long-term outage where I could see if the generator could cool the whole data center. That's a question mark for me. That's just an issue I haven't been able to test. How do you test that in real life? Shut down the power and wait to see what happens?"
The servers Allen has identified as running high-priority applications receive power from the two CoolAir UPS units. Since these UPS units also produce cold air as a byproduct, Allen is able to use that cold air as a supplement to his data center cooling system that may be running at a suboptimal level when on generator power.
Allen said he wants the extra cooling insurance CoolAir UPS technology offers because he's seen what can happen when his data center starts to run hot. And some of his servers can't handle the emergency shutdowns that occur when they do run hot.
"I've had this room get all the way up to 85 degrees," he said. "We've had failed hard drives, machines that refused to come back up again. I've had a lot of corrupted data because the machines had to shut off in hurry. We have one machine that takes 30 minutes to shut down. It's a million dollar machine. But it has a built-in shutoff. If it gets hot, it shuts off. That leaves you in a situation."
Let us know what you think about the story; email firstname.lastname@example.org.