SAN FRANCISCO -- Part of the excitement around platform as a service -- the cloud computing model in which a third-party...
provider delivers hardware and software tools -- is that PaaS platforms make it much easier for enterprises to quickly spin up experiments in a safe and secure way.
But the computing innovation also comes with many challenges, tech executives from Deutsche Bank and Experian explained at the recent Red Hat Summit -- from getting IT staffers to buy in, to diffusing distrust between team managers, to managing shadow IT innovation.
At Deutsche Bank, Tom Gilbert, global head of PaaS technology, said bank executives saw clear benefit in being able to spin up new infrastructure more easily and wanted to get to 80% adoption of the Red Hat OpenShift platform. But mandating this new way to work was "not the way" to go, he said.
"You cannot get there by forcing that," he said. IT teams were wary of PaaS platforms, he said, fearing they would lose ownership of the servers they had spent years building. "So, we set out to have a carrot-flavored stick, where you are rewarded for doing the right things."
That began by emphasizing the positive, including the fact that IT teams could reduce the time to provision a new server from six months to a few days. This meant they could test out a new idea in production in three weeks, and if the idea didn't pan out, it was not such a big deal -- they could try out a new idea.
Encouraging risky communication
One of the challenges with moving faster, however, is developers might be inclined to pursue riskier ideas. Gilbert said banks, by virtue of what they do, spend a lot of time thinking about managing risk for clients. Now, with PaaS, Gilbert and colleagues were also spending more time managing risk, as it related to a faster pace of development. These included risks related to IT security, to delivery and team dependencies.
Dependency risks can act as a barrier to collaboration, especially when a team manager's job is on the line. "If I have a dependency on your project, I may be more inclined to do it myself," Gilbert said, to minimize the risk of it being done wrong. As a result, teams end up duplicating effort. "It's one of the hardest things to tackle."
One way to increase collaboration is to encourage team managers to frankly discuss the types of risks associated with new PaaS projects. The more information that is available, the more confidence managers can have in taking a measured risk, which in turn leads to more trust and collaboration. It helps to open up the risk logs -- the master documents created in the early stages of a project to track issues, Gilbert said.
Shadow innovation or 'free problem-solving'
Another side effect of a risk-averse culture is developers sometimes pursue great ideas in secret. "This risk-averse culture sometimes keeps people under the radar when experimenting," Gilbert said.
Tom Gilbertglobal head of PaaS technology, Deutsche Bank
For example, when the bank first started building out its PaaS infrastructure, it focused initially on stateless apps in which data is kept outside the app itself. This precludes deploying things like databases on to the infrastructure. When Gilbert was visiting the bank's Moscow office, he told the PaaS team about the long-term roadmap to develop tools for stateful databases. They told him that Dimitry, one of their programmers, had been experimenting with this for about three weeks, and it turned out to be pretty good.
"It was shocking and surprising. And in an earlier phase of Deutsche Bank, that would have been slapped down, since they were not database engineers," Gilbert said. Instead, the team was publicly celebrated, and their names were woven into the code, which is now widely used by the bank.
"Now, we are trying to reward what is effectively free problem-solving in the organization," Gilbert said.
Promoting an experimental mindset
Experian's initial approach to PaaS platforms started as an experiment when the credit reporting agency found that its current solutions for deploying applications using virtual machines were a major bottleneck to new projects.
Jon Deeming, vice president of the PaaS center of excellence at Experian, helped build a small PaaS pilot, which eventually got rolled out across the organization.
Management also had to contend with cultural fears related to adopting PaaS, Deeming said. Each engineer dealt with the change differently, passing through the various stages of awareness -- from disbelief to denial and, finally, acceptance.
"The denial phase can be hard, and there can be a lot of pushback," Deeming said.
But he said some were quick to recognize the benefits of a more fluid infrastructure. For example, quality-assurance teams immediately saw the opportunity to dramatically reduce their testing times in PaaS by running tests in parallel. Other teams seemed upset when they couldn't use the same tools to monitor their workloads. "You have to look at each team in terms of how you manage them," Deeming said.
It's important to encourage new ideas, but also manage expectations. "You want to tell people that, in 90 days, I want 10 good ideas, but I will kill nine of them," Deeming said. Setting expectations spurs innovations, but in a way that communicates there may not be enough resources to pursue all the good ideas -- at least for the moment.
Cultural norms, video conferencing
One of the challenges of managing an IT infrastructure across the world -- whether the infrastructure is PaaS or part of a traditional data center -- lies in recognizing and responding to the cultural norms of different teams. Experian has used various culture analytics tools to help get a pulse of how teams can vary across countries. For example, low-conflict cultures require a lot of coaching, which can be exhausting, Deeming said. Other teams like more direct communications, which makes it easier to talk about problems in a productive way.
One good practice Deeming recommended when adopting PaaS platforms is to mandate video conferencing meetings, rather than phone calls. In the U.S., there is a habit of putting the phone on mute and working on other things during the meeting, but this just wastes everyone's time.
Video also makes it easier to pick up on nonverbal cues, such as someone sitting back, crossing their arms or frowning. Deeming said these nonverbal cues signal something is amiss, and they can help managers draw out an important technical concern that may not have been obvious to the rest of the team.
Bringing innovation to operations
IT operations teams also have slightly different concerns around innovation than development teams do. They tend to be more risk-averse, as they tend to be on the frontlines when failure occurs. Rather than innovating on new apps, they tend to focus on innovating around change management processes.
Rob Sexton, vice president of IT operations at Experian Health, said he is a fan of process innovation. He encouraged his staff to experiment with different approaches, but to put everything into a shared change management system.
One strategy to encourage responsible experimentation on PaaS platforms involves giving IT managers and their individual teams ownership of different tools. But when a problem occurs, it's all hands on deck, with the whole IT organization deputized to jump in to solve the problem, even if they are up for 55 hours, Sexton said.
At the same time, it is important to give staff guidelines for experimentation to discourage people from accidentally deploying new tools with vulnerabilities or malware. "We have great security polices to prevent a lot of problems, but at the same time, we don't want to squash a good idea," Sexton said. This mindset helped drive the adoption of a variety of tools, such as chat services to improve incident response.