This content is part of the Essential Guide: An IT security strategy guide for CIOs

Essential Guide

Browse Sections
Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

'Threat hunting' emerges as new enterprise security role

Enterprise security is often about the latest technology fix, but some experts believe personnel cast in a threat hunting role could provide a valuable line of defense.

As new waves of cybersecurity attacks threaten organizations, the typical response has been new layers of technology and automation. But some IT security executives believe the current environment calls for a new category of employee: the threat hunter.

The threat hunter's mission, proponents say, is to find the security incidents automated systems miss. This "unto the breach" calling aims to strengthen security operations centers (SOCs), which serve as the focal point for an enterprise's threat detection and incident response activities. SOC modernization has become a more pressing issue of late, as organizations revisit their IT security nerve centers.

Ed Amoroso, the former chief information security officer (CISO) at AT&T and now a security consultant, said SOCs have progressed over the years from "a bunch of people in cubicles" to a more seamlessly integrated blend of manual and automated systems. The threat hunter is part of that evolution, he added, speaking at a recent SOC webinar sponsored by Sqrrl Data Inc., a company that provides a technology platform for threat hunting.

"I don't think you can ever get away from [the need for] really intelligent people who dig through data to find out what is going on," Amoroso said.

CISOs, who typically staff and operate an organization's SOC, face the challenge of hiring and managing scarce threat hunter talent. But they will need to do just that to reach the highest levels of preparedness.

Without the threat hunters, it's very difficult to be anticipatory.
Johna Till JohnsonCEO, Nemertes Research

Johna Till Johnson, CEO of Nemertes Research, agreed with the need for threat hunters, which she described as part of having a mature security operation. Nemertes uses a four-level security model to benchmark an enterprise's security posture. The model's lowest level, zero, indicates that an organization is unprepared, while level one designates "reactive" security. At level two, the organization is "proactive" and becomes "anticipatory" at level three, she explained.

"Only about 10% of the security organizations we talk with operate at the anticipatory level, but part of being anticipatory is preparing for threats that don't yet exist, or are newly emerging," Johnson said. "Without the threat hunters, it's very difficult to be anticipatory."

Characteristics of a threat hunter

David Bianco, who was lead threat hunter at General Electric prior to becoming lead security technologist at Sqrrl, describes threat hunting as the use of manual or machine-assisted techniques for detecting security incidents that an organization would otherwise not know about. The knowledge gap, he said, stems from the failure of automated systems to detect a particular threat or the lack of systems designed to detect certain types of activities. As for the former, Bianco said automated systems are great at finding automated threats, such as mass- market Malware. 

"It's when you pit automated detection against skilled human actors that the biggest problems occur," he said.

So what types of skills should a CISO look for when hiring a threat hunter?

characteristics of a threat hunter

For one, threat hunters need to live on the front lines of security research, attuned to the latest and most dangerous emerging threats, Johnson said. That role, she said, means the hunters need access to key tools and services, such as behavioral threat analytics and "threat intelligence networks and solutions."

But for Johnson, the most important job requirement is creativity.

"A threat hunter needs to be able to put his or her mind to the task of uncovering vulnerabilities that are specific to the organization, and may or may not be highly sophisticated," she said.

For example, a security vulnerability could be as prosaic as a door left open in a facility to enable smokers to take a smoking break, she said.

Amoroso suggested threat hunters need to have a solid grasp of the corporate routine and the ability to seize upon any departure from the norm. In addition, he said hunters should be ready to look for trouble in increasingly complicated IT environments, which he characterized as becoming much more distributed, virtualized and automated.

"Hunters need to figure out how to jockey this," Amoroso said.

Bianco, meanwhile, cited critical thinking, business knowledge, communication and collaboration as important hunter skills.

The trick for CISOs is finding people with the requisite skills.

"Today, the issue is it's just kind of hard to find people who are good at this," Amoroso said.

Setting up a hunter team

Enterprises may opt to grow their own hunters from the pool of security analysts already working in a SOC. Bianco said organizations structure their hunter teams in different ways. For example, a SOC director might ask security analysts hunt threats in their spare time. The trouble with this ad hoc approach is that spare time is in short supply in the typical SOC, he said.

On the opposite end of the spectrum, an organization might build a dedicated hunter team. That approach has the advantage of focus, but it comes at the cost of concentrating all of a SOC's hunter skills in one group of people. A better approach, Bianco suggested, is to create a dedicated hunt function in the SOC, but make some or all of the hunter positions temporary assignments. With this hybrid approach, more people gain hunter experience as they rotate in and out of the hunter role. They also retain those new skills when they go back to their regular assignments.

Once a team is in place, the next task is management -- more art than science at this stage of development.

"Nurturing expert hunters today is in its infancy," Amoroso said. "[The task] does require a management team ... that 'gets' the psyche of the hunter."

Hunters are, by necessity, highly intelligent, Amoroso said. Adept at plucking out things that don't make sense, they will readily identify any management nonsense, he suggested. Managers should create an environment where hunters can focus on the difficult task at hand, without having to contend with too many workplace constraints and conventions -- such as wearing a suit.

"A lot of BS from management is not going to work. The antennas for detecting any type of silliness from management are very high," Amoroso said.

Amoroso added a large part of management's role is making sure threat hunters have the three key things they need to do their jobs: access to data, appropriate tooling and training/professional development.

Hunting: Potential drawbacks

Michelle Drolet, CEO of Towerwall Inc., a Boston area data security services provider, said the threat hunter role could be valuable, noting that paying more attention to information security and making it a priority helps thwart intruders.

On the other hand, she has some reservations about threat hunting. Repetition is one concern. A threat hunter that looks at the same information and the same systems, over and over again, could become complacent over time. Drolet likened the situation to an organization having its own penetration testers repeatedly probing the same applications.

"They get used to them," she said. "Do you need that second set of eyes?"

In addition, Drolet questioned whether an organization's IT managers will take a threat hunter's findings as seriously as a written report from a third-party security reviewer, documenting security vulnerabilities and providing a remediation roadmap.

"Is the CIO or CISO going to prioritize those vulnerabilities," Drolet said of a threat hunter's findings.

Next Steps

Find out how CIO/CISO reporting structures should be modified

Read a Q&A featuring MGM Resorts' CISO

Learn about the need for CISOs and CIOs to collaborate more closely together 

Dig Deeper on IT staff development and retention