A botched IT outsourcing contract in the state of Virginia has led to missed deadlines, widespread service outages in several state offices and even the firing of the state's CIO -- and holds lessons for any organization outsourcing IT operations.
Virginia's $2 billion IT outsourcing contract with global security firm Northrop Grumman Corp. has resulted in numerous network crashes and uptime difficulties, according to a legislative audit released Oct. 13. The contract covers a computer upgrade and management of the state's IT systems and services. (See sidebar for Northrup Grumman's response.)
But while state workers and residents have felt some ill effects from the IT outsourcing arrangement, terminating the 10-year contract could cost the state as much as $400 million and leave Virginia with nobody to manage its computer systems, reports say. Moreover, the initial contract, signed in 2005, did not include sufficient penalties for poor service levels -- hence, the state cannot collect monetary damages related to these IT interruptions.
The situation is an extreme example of IT governance failure, outsourcing experts say. Yet it's a reminder that IT outsourcing mistakes can happen in any organization -- public or private, large or small -- "where there's not the right governance or maturity, and there are systemic challenges in outsourcing," said Helen Huntley, a research vice president and IT outsourcing analyst at Gartner Inc., a Stamford, Conn.-based consultancy. "It sounds like some generic bad outsourcing behaviors on each side."
Don Flores, partner and director of state and local government services at TPI, a global sourcing advisory firm based in Houston, said CIOs at all types of organizations should learn from Virginia's difficulties because they could happen anywhere.
"These situations you see in Virginia likely happen in the private sector; you just don't see the scrutiny of it," he said.
As Virginia's woes make all too clear, any organization planning to outsource IT operations needs an effective governance plan, with clear service-level agreements and penalties spelled out in the initial agreement.
Virginia's IT challenges date to 2003, when the state first sought a private company to oversee its computer system. At that time, as The Washington Post reported, the state decided to employ a governance structure in which an independent board -- not the state's governor or general assembly -- hired Northrop Grumman and was put in charge of the state's technology agency, the Virginia Information Technologies Agency (VITA).
The arrangement led to confusion and a lack of oversight, which infelicitously dovetailed with service failures by Northrop Grumman that disrupted public service. The Post reported that it took Northrop Grumman six months to provide computer and phone services to 17 temporary unemployment offices, rather than the one to three months that a state economic commission had sought.
Elsewhere, a Department of Motor Vehicles office lost network connection for 31 hours this summer, forcing customers to reschedule their appointments, and a Department of Environmental Quality office was without phones or computers for more than a day in May.
The contract originally called for Northrop Grumman to assume full control of the state's computer infrastructure by this past July 1, but the audit indicates that the company has missed numerous deadlines and fallen behind schedule.
In June, the board fired CIO Lemuel C. Stewart Jr. after he tried to stop a $14 million payment to Northrop Grumman. In August, a new CIO, George Coulter, was hired to oversee the state's IT strategy.
Lessons learned: Governance, SLAs needed in IT outsourcing contracts
Both VITA and Northrop Grumman missed opportunities to set this arrangement right from the start by completing due diligence, establishing proper IT governance and crafting service-level agreements (SLAs) with quick remedial action guidelines, experts interviewed about the issues say.
Gartner's Huntley questioned whether Virginia effectively understood and communicated the thinking behind a consolidated outsourcing environment to its users.
"Did the state of Virginia truly understand the agency needs and the complexity of the environment?" she said. "In outsourcing, you need to understand the value brought to the table and communicate and mandate this change throughout the enterprise. In this case, I'm not sure if there was some resistance from the downstream agencies."
VITA also should have been clear on who controlled the vendor relationship, who had legal accountability, and who spoke on behalf of the client to the outsourcer, Huntley said.
"Outsourcing deals need strong vendor management and governance of the outsourcing relationship and, in this deal, I'm not sure if there was a lack of accountability on the client's side of the outsourcing deal," she said. "When there's not a clear line of demarcation about who owns [the deal] and who speaks on behalf of the client, it gets to be challenging in [determining] who is governing the deal."
Most IT outsourcing deals include performance metrics, typically in the form of SLAs, whereby each service level has an associated penalty, Huntley said.
"It's one thing if the provider is not delivering -- that's a breach of contract, and then you have a right to get out of the deal," she said. "I don't need clients to put a separate metric on every step of the way, but they need to manage the outcome of what the vendor delivers. Otherwise, service level is subjective … and if they have service levels without penalties, it's subject to the vendors."
TPI's Flores said his organization advised the state of Georgia in an IT outsourcing deal, and spent a full year designing its governance team, training people and even putting them through role-playing scenarios to ensure that everybody knew his responsibilities.
"I don't know if VITA did that or not, but I look at the results and think it looks like they didn't," he said. "The client side really has to be well positioned to govern and manage this new way of receiving services and, as an interim agency, having those services delivered to the state."
The audit recommended that Gov. Timothy M. Kaine be placed in charge of the agency, which Flores seconded. "You can't really choreograph well with Northrop Grumman without the governor's direct oversight of it," he said.
The contract included a pricing cap, but Flores said the amount might not have been realistic and set the vendor up to fall short.
"In some ways, this contract was structured to really favor the state, and that kind of puts Northrop Grumman in a natural defensive position," Flores said. "They're there to make a profit and they have a cap, so they're going to try to not run their costs past the cap."
An effective IT governance framework should trump all other concerns in the crafting of an IT outsourcing arrangement, Flores said.
"It's hard enough to strike these contracts, but it's even harder to manage them afterwards," he said. Still, "great governance can manage a bad contract, but a great contract cannot guarantee success -- you have to have good governance for it."
This was first published in October 2009