Skip to content
Back to blog

Intelligence

Security Investigators Have to Become Data Scientists

Alexander Snyder6 min

The traditional security investigator receives a report, reviews the evidence, builds the case, and closes it. The workflow is linear: incident occurs, investigation opens, evidence reviewed, subject identified, case closed.

This workflow made sense when evidence was sparse and the investigator's job was to find the needle in the haystack. It's the wrong workflow when evidence is abundant and the investigator's job is to understand what the haystack is telling them before any specific needle appears.

The transition every serious security operation is going through isn't about learning new software. It's about changing the fundamental mental model of what investigation looks like.

The reactive model and its limits

The reactive model is built around known events. Someone reports a loss. Security is notified. The investigation begins with what's known and works backward: who was present, what cameras covered the area, what access records exist.

This model catches what gets reported. It finds what it's pointed at.

The problem is that the most significant security events, organized theft rings, systematic fraud, insider threats operating over long periods, don't announce themselves. They accumulate. A single incident looks like a minor loss. Ten incidents spread across locations over six months looks like a ring. The reactive model sees each incident as a closed case. It never sees the ring.

What data science changes

A data-first approach doesn't start with an incident. It starts with questions: what does normal look like, and what deviations from normal are worth understanding?

In retail security, normal is the baseline for cycle count variance, EAS activation rates, sales patterns relative to traffic counts, and staff scheduling relative to shrink. Deviations from that baseline might be explained by legitimate variation, seasonal patterns, staffing changes, product mix shifts. Or they might be explained by something else.

The investigator running a data-first approach isn't waiting for a reported incident to ask "what happened here?" They're running continuous queries across their data and asking "is anything here worth understanding?" The cases that emerge from that question are often different from the cases that come in through the intake form.

The specific win that changes how teams think about this: a product category shows unusual shrink for two months, no incidents reported. The investigator looks at EAS activation data, POS audit data, and traffic counts. A pattern emerges. Three stores, similar timing, similar product mix. Before any incident report exists, they have a hypothesis. When an incident is finally reported, the investigation doesn't start from scratch. It confirms or refutes a theory that was already months in development.

That's a different kind of work. It requires different habits, different tools, and a different relationship with uncertainty.

The skill gap

Most security investigators are skilled at evidence analysis. They're trained to build a case from what they have toward a conclusion. They're less skilled at exploratory analysis, working through data to generate hypotheses before they have a conclusion to work toward.

These are genuinely different cognitive skills. One starts from a known outcome and works backward. The other starts from open questions and works toward patterns that may or may not indicate anything significant.

Developing exploratory analysis skills requires:

Comfort with ambiguity. Exploratory analysis doesn't produce a case. It produces hypotheses of varying confidence. An investigator trained to close cases can find this unsatisfying. "I looked at all this data and I'm not sure what it means" feels like failure when the mental model is case-based. It's not failure. It's the correct output of an exploratory pass.

Understanding of what data the organization has. This sounds basic but is genuinely hard. Security operations often have access to far more data than they use: EAS activation logs, people counter data, access control records, POS audit logs, fleet GPS records, communications records. Much of this data is collected for other purposes and never looked at from a security lens. Knowing what data exists and what questions each dataset can answer is a prerequisite for data-first investigation.

Ability to work with imperfect data. Operational data is messy. EAS activations include legitimate alarm events. Traffic counters have calibration drift. Sales data has legitimate variance. A data-first investigator needs to understand the noise floor of each dataset before they can identify signals worth chasing.

Where AI fits

AI doesn't replace the investigator's judgment. It changes what the investigator's judgment is applied to.

The specific application that's generating the most traction: using AI to surface anomalies in large datasets that a human analyst working manually would miss or would take weeks to find. The AI identifies the pattern. The investigator evaluates whether the pattern means something and decides what to do about it.

The second application: connecting incidents across time and location that share characteristics but would be filed as separate cases. An organized ring leaves traces in multiple incident reports that individually look routine. AI-assisted pattern matching across a large case database surfaces the connections. The investigator then decides whether the connections indicate an organized operation or coincidental similarity.

Neither application works without investigator judgment. The AI surfaces possibilities; the investigator evaluates them. But the combination can address threats that the purely reactive model can't see.

Making the transition

The organizations doing this well have not told their investigators to "use AI." They've changed what investigations are evaluated on. Cases closed is a metric for the reactive model. Cases initiated from proactive data analysis, ratio of proactively identified incidents to reactively identified incidents, time-to-pattern on organized activity. These are metrics for the data-first model.

What gets measured determines what skills get developed. If investigators are evaluated on case close rates, they'll work cases efficiently. If they're evaluated on proactive identification, they'll develop the skills to identify things before they become obvious.

The transition doesn't happen by adding a new tool to the existing workflow. It happens by changing what good investigation looks like.


PurviewX builds intelligence platforms for security-intensive industries. Start a conversation.