This content is part of the Essential Guide: Project management strategies that transform businesses: A CIO guide

Essential Guide

Browse Sections
Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Hopper: Crunching big data, making travel easier

In the Cambridge, Massachusetts, high-tech Kendall Square district, the Hopper offices hardly stand out from the outside, tucked away in a brick building that used to be the old Kendall Boiler and Tank Co. building. But this high-tech travel company, started six years ago by three former Expedia engineers with the aim of making traveling cheaper and more fun, has been making quite a name for itself lately. Following a New York Times article this spring touting the company's new customizable tools to help consumers lower their travel costs, the 26-employee business has been featured in media outlets ranging from the Boston Globe and Los Angeles Times to Good Morning America and CNN Money.

SearchCIO recently caught up with Hopper's Chief Data Scientist Patrick Surry to inquire about the big data technology that fuels Hopper's business model. It's a list of open source-only software that includes everything from mainstays such as Hadoop, a distributed computing framework for large data sets, to relative newcomers such as Spark. A large-scale, in-memory data processing engine, Spark is only beginning to gain momentum, but Surry and his team are already experimenting with it to see "if we can improve that ad-hoc analytical cycle," he said.

Given the company's relative longevity in the here-today, gone-tomorrow startup world, we also asked Surry about the workplace culture that's allowed Hopper to attract the IT talent required to get a big data startup off the ground. Surry, who joined Hopper in April 2013 after a two-year stint at Pitney Bowes, said it takes a certain type of person to thrive in what he described as a "challenging" environment. Imagination is important, but so, it seems, is a thick skin. Here's what he had to say:

"The qualities we look for in employees are all about being flexible, obviously creative and willing to fail fast. A lot of what we do at Hopper is figure out what the right way to position and deliver the solution to the problem is. It's challenging -- we build stuff, we throw stuff away and then we build new stuff.

"It requires a certain kind of attitude, I think, among the developers. You have stuff you've worked on for three months, and then we decide to throw it away and do something different. That can be frustrating for some people. And I think for others, that's part of your traction.

"I think a lot of companies get bogged down because you've created something that sort of works and you have to continue to maintain it forever. I think as a startup you have the luxury of saying 'Hey, that doesn't work. Let's try something else,' both from a kind of business point of view but also from an infrastructure point of view."

Interested in seeing that mentality in action? In this SearchCIO video, Features Writer Kristen Lee gets a VIP pass to Hopper offices.

Read more about how Hopper is experimenting with big data tools in Apache Spark's in-memory processing is turning heads.

View All Videos