News Stay informed about the latest enterprise technology news and product updates.

The problem with focus groups

Running a focus group is not unlike gathering requirements for an IT project: Both rely on collecting qualitative information to tap into the customer’s perspective. As IT professionals know, what business users say they want and claim they do often differs from what they really need or actually do. Focus groups also fail to tell the whole story, as Farrah Bostic, founder of The Difference Engine in Brooklyn, pointed out during her talk at Strata + Hadoop World in New York City.

Bostic doesn’t consider the focus group a broken tool; it’s more like it’s become a crutch. Instead, businesses should look beyond the focus group — or any one tool — to collect qualitative research. “One of the problems I see is that we use one tool, and often not the right tool, to figure out what we’re going to do next as opposed to using all of the tools at our disposal,” Bostic said.

Several years ago, in a study Bostic did with a food marketer, she asked customers to talk about how they put together their grocery lists and how they did their grocery shopping and then asked them to record the process with a video camera.

One customer said she didn’t need to make a list, and that it was all, “up here.” That turned out not to be true, based on the video evidence, which showed her opening every cabinet and both refrigerators to find out what she was running low on and then jotting down a list of what she needed.

Even more telling? Only 50% of what the customer included on her list ended up in her grocery cart, and the other 50% was swapped out on the fly for something that looked better, for something new she decided to try or with an “artifact” gathered by her children, Bostic said. “What happens is that you encounter reality,” she said. “She placed a bet on what she was going to buy … and was right about 50% of the things she bought.”

The big takeaway for Bostic in this study was how the combination of having the customer describe the process and observing the process revealed what the customer valued, what the customer believed about herself and how those values and beliefs played out in “real” life. “If we just had focus groups, we would have just heard her tell us she doesn’t make lists,” Bostic said. “And that would have been an incomplete picture of her life.”

For IT professionals who often find themselves frustrated by the standard requirements gathering process, job shadowing might be a useful tool. Rather than just rely on business users to tell you what they think they need, watch how they work and derive or at least modify requirements based on their actions.  As the late great Yogi Berra said, “You can observe a lot by just watching.”