When Right is Wrong

In my recent post How to Get the Wrong Answer, I stated that analytic correctness (e.g. the “right answer”) increases when large volumes of data are highly varied.

Analytic correctness also has a corollary statement: the right answer is wrong if it takes too long too calculate.

A good example of this corollary can be found in high-tech security. Consider the chart below, which depicts the evolution of an intrusion, and the criticality of reducing the amount of time the intrusion goes undetected:

This slide was adapted from a report jointly created by the North American Electric Reliability Corporation and the U.S. Department of Energy. The report describes how sophisticated intruders are creating Advanced Persistent Threats (APTs) that penetrate and persist within corporate firewalls (page 32):

Advanced Persistent Threats (APT) are becoming a significant concern across all sectors. These threats involve sophisticated, determined, coordinated attackers who systematically compromise government and commercial computer networks. These attackers typically install multiple backdoors into a cyber network they are attempting to infiltrate, under the “radar” of even the most sophisticated anti-virus protections, thereby establishing a secure foothold into the network. They then install utilities to exfiltrate data to external servers. Attackers respond to attempts to eradicate infection and remediate network security by establishing additional footholds and improving sophistication. These infiltrations can persist, untraced, for months and even years.

Given this backdrop, one obvious strategy is to identify and address intrusions as quickly as possible. Many corporations rely on SIEM technologies and solutions(Security Information and Event Management) to accomplish this objective. These solutions traditionally provide real-time analysis of security alerts in order to detect intrusions.

Some of these approaches, however, are running out of gas. Traditional SIEM analytic tools no longer supply the right answer fast enough. Why?  Because the old approach lacks variety. The old approach relies too much on one form of data: security logs. Log-heavy analytics run the risk of either missing the intrusion or taking too long to identify it. IT operators in a Security Operations Center (SOC) may try and augment the security logs with other forms of input, but their SIEM infrastructures either weren’t designed for that volume of data, or they weren’t designed to handle massive varieties of streaming data.

A re-think of SIEM infrastructure was required and a new, innovative approach became necessary.

Chuck Hollis did a great job summarizing a new approach for security analytics in the data center. The model capitalizes on new data center architectures that I have written about previously: the ability to analyze massive amounts of recent, streaming security data alongside of a deep historical archive.

Instead of a log-heavy approach, this new style of SIEM architecture accepts a massive variety and volume of data. In addition to traditional security logs, these new repositories also contain complete firewall data,
network configurations, operating system state, a complete list of data center assets (e.g. a CMDB or configuration management data base), and network traffic traces.

The system requirements for keeping track of this variety of data will challenge a SIEM architecture designed for fewer sources of information. Capturing all network traffic, for example, is a wildly different use case than collecting security logs from devices.

The new approach espouses massive variety. Analytics running on top of this amount of variety have a much better chance of distilling out indications of attacks, threats, and vulnerabilities. Analytic models that run on top of these new architectures can take into account streaming and real-time data, as well as deep historical data.

Converged data center architectures, in which customers choose to buy pre-packaged cloud infrastructure (e.g. VCE), become a more secure choice. All components in the converged platform can pre-integrate to work in unison and cooperation with this new type of security analytics platform.

Does this mean that data center architectures that “mix and match” cloud infrastructure components are inherently less secure? Not necessarily. There has been recent innovation in this area that I will explore in a future post.

image credit: emc.com

Wait! Before you go…

Choose how you want the latest innovation content delivered to you:

• Daily — RSS Feed — Email — Twitter — Facebook — Linkedin Today
• Weekly — Email Newsletter — Free Magazine — Linkedin Group

Steve Todd is an EMC Fellow, the Director of EMC’s Innovation Network, and a high-tech inventor and book author Innovate With Global Influence. An EMC Intrapreneur with over 200 patent applications and billions in product revenue, he writes about innovation on his personal blog, the Information Playground. Twitter: @SteveTodd