Disaster prevention and mitigation strategies: Strike early and often
A comprehensive collection of articles, videos and more, hand-picked by our editors
"Disaster recovery is dead. I think it's been dead for a long time," Alan Berman was saying.
It might seem odd that Berman, executive director at the Disaster Recovery Institute International and former CIO at a major financial institution, should proclaim that information technology disaster recovery is dead. But the point was not to deny disasters -- they happen. Rather, that disaster recovery (DR) and business continuity have become synonymous.
"We don't do recovery anymore, because what everybody wants is continuous operations," Berman said.
While there are probably industries that still take 6,000 tapes to a recovery center and restore all the data, he said, restoring thousands of severs comes fraught with mishaps. The CIO's worst nightmare is corrupted data. "When I'm talking to CIOs, it is all about how do we make sure the data has integrity," he said.
Continuous backup also removes the burden of disaster recovery testing, which many organizations give short shrift.
"Organizations swap from their primaries to their backups without even thinking. You can go over and make sure the backups are good five times a day, and organizations do that," Berman said. "We're not really trying to retrofit recovery into the operation; it is the operation."
Berman is not the only one making apocalyptic statements about DR. Server and storage virtualization -- done internally and provided externally in the cloud -- have fundamentally changed the kind of equipment and processes needed for recovery by companies and by disaster recovery outsourcing providers.
The use of cloud computing is effecting a "tectonic shift" in DR, said Sam Gross, vice president of global IT outsourcing solutions at Unisys Corp. Companies can now use cloud computing to conjure copies of their applications that can be brought to life at a moment's notice. And companies that are building new data centers, like Yahoo Inc. and Google Inc., are constructing them so that any data center can stand in for any other.
"The cloud allows you to plug in anyplace," Berman agreed, citing a recent demonstration of this in communications, where he watched a company eliminate the need for employees' desk phones by spoofing their cell phone numbers to look like calls were coming from the original source. "Technology is moving toward continuity. We are becoming far more time sensitive than we ever have before, and that has changed recovery."
Yahoo's greening of the data center
In 2005, Yahoo Inc. housed its data centers in colocation facilities around the globe. Like just about every other company whose business depends on IT, Yahoo put those centers in the "NFL" cities of the colocation industry -- San Jose, Calif.; Chicago; the D.C. suburbs. In fact, the Sunnyvale, Calif.-based online portal had a data center in every major-league colo city on every continent except Africa and Antarctica.
These facilities were built to "grenade-proof" specs. But there were drawbacks, said Scott Noteboom, Yahoo's director of data center operations. Power tended to be more expensive in these locations, and the centers were inefficient.
"Most of the power was for cooling and inefficiencies, with no focus on clean power," recalled Noteboom, a speaker at the recent EmTech conference at MIT. The cost to cool -- calculated as the percentage of server power required to cool the plants -- ran between 35% and 55%.
Yahoo decided it could do better building its own centers. Noteboom is now on his fifth generation of data centers.
"Every single project has been a new generation design," he said.
A building that used to take a year to build now takes six months. Much of the facility is pre-engineered. Yahoo's strategy today is to build in "tier two" markets, scouting the globe for places where the weather is better and power is cleaner and cheaper.
Evaporative cooling technology has done away with refrigerants. At Yahoo's latest center in upstate New York, the entire building functions as an air handler, taking advantage of the winds off the lake to eliminate chiller plants in the data center. The average cost to cool for Yahoo centers is now 1%. -- L.T.
Internet service provider Savvis Inc. espouses the same view. The company is building its cloud computing infrastructure for business continuity to be a DR substitute, "for pennies on the dollar," said David Shacochis, vice president, research and development.
Logistics: Distancing the technology from the business operations
Despite the fact that IT now has cloud computing and storage area networks and communications systems that can plug in anywhere, many companies still insist on putting the technology under the same roof as the business. But that, too, is beginning to change, Berman said.
A few years ago, he was consulting with a big financial institution that complained of frequent outages in critical servers. The company housed the data center with the business operations, "because the operations felt they had to touch the technology." It turned out that when members of the nightly cleaning crew mopped the floors, they were short circuiting the servers.
Berman said the notion of physically distancing the servers and data center technology from business operations is an offshoot of the British "triangulation" method that basically advocates taking the technology that runs one building's operations and putting it in a different building. "People are getting smarter about not putting everything at risk at the same time," Berman said.
Yahoo rethinks business continuity
Yahoo has spent the past four years reinventing its business continuity and disaster recovery strategy as it built new data centers. In its newest data centers, Yahoo is eliminating uninterruptible power supplies (UPS) and backup generators by ensuring that every data center can stand in for any other.
When the lights go out at Yahoo's new upstate New York center, another facility resumes the work, said Scott Noteboom, director of data center operations at Sunnyvale, Calif.-based Yahoo. The result is high-performance geographic replication of functions.
"The redundancy is now between markets in a country, not in the data center itself," said Noteboom, who was recruited by Yahoo from the colocation industry to head up the company's new data center strategy and talked about it at the recent EmTech conference (see sidebar).
Yahoo's motivation for eliminating the UPSes and generators was to cut the power losses that come with this gear. The move has reduced energy losses due to power transmission in the data center by half, from 38% to 19%. And that leaves some serious money to plow back into backup and replication between the data centers.
Mark Schlack also contributed to this article. Let us know what you think about the story; email: Linda Tucci, executive editor.
Recession squeezing IT disaster recovery budgets
Technology is changing IT disaster recovery outsourcing
A disaster recovery plan for branch offices: Five layers of redundancy
When IT disaster recovery plan is put to the test, VoIP becomes savior
Business continuity standards, certification get legal push