By: Kamran Ahsan

Intelligence Driven Security

Using big data analytics to power information security gives business environments the ability to find a “needle in a haystack”

As IT evolves and becomes more advanced, the methods employed to carry out cyber attacks follow suit. This brings a significant change in the profile of cyber security risks. As such, organisations need to change the way they assess, prevent, detect, and respond to advanced cyber threats. Managing these sophisticated threats, given the quantity of security data being generated and acquired by organizations, can be equivalent to finding a needle in a haystack. This challenging situation can be dealt by sifting through the huge chunks of data and extracting meaningful information out of the same. Big Data analytics offers this potential to drive security through intelligence and enables business owners to make informed decisions.

The Rise of Big Data and the Attack Surface
We witness an era of data explosion. Multitudes of applications, which are accessible through various channels, from anywhere and at anytime, are generating data. The data is generated at a phenomenal speed, is big in volume and is rich in variety. Data with such attributes is termed as Big Data. The Merriam-Webster Dictionary ( defines Big Data as “an accumulation of data that is too large and complex for processing by traditional database management tools”.

To give you a better idea of what this exactly means, let’s look at Intel’s view on “What Happens in an Internet Minute?” (as viewed on Aug 01, 2014) (1):
• Around 1.57 million gigabytes of IP data is transferred
• 347,222 Tweets are sent via Twitter
• 6.9 million messages are published on Facebook
• 4.1 million searches are initiated on Google
• 138,889 hours of videos are watched on YouTube

These statistics are experiencing an upward trend and these numbers might no longer be valid as you read through this article. Not only we witness people to people and people to machine communication but also machine to machine information exchange to enable concepts of smart areas, neighborhoods and cities. Big Data is here to stay. As it grows, it needs to be managed – acquired, stored, analyzed and utilized for business benefit. Big Data utilization through analytics has resulted in strides in numerous areas including customer behavior and priorities, marketing campaigns, product positioning, science, healthcare, sports and stock trading to name a few. Big data analytics helps to reveal unidentified patterns and connections and gives important insights to influence future activities related to the data being analyzed.

From an information security perspective, this high-volume, high-velocity and high-variety data represents an alarming situation – the attack surface is getting bigger and wider. The spectrum and nature of cyber threats increase the likelihood of data compromise and subsequent misuse affecting individuals, businesses and even countries.

Information Security Management and Layered Defense
This alarming situation diverts our attention to look back at information security fundamentals and analyze what we have as part of our arsenal. Information security management demands for keeping a fine balance between threats & vulnerabilities, and security controls.

Accordingly, information security stakeholders constantly strive for achieving this balance to ensure commensuration of worth of information assets and associated cost of security controls.

Information security stakeholders are also acquainted with “the cyber battlefield” where target capabilities are at constant risk of being compromised from a variety of attackers. As shown in the table, the defense zone forms a buffer between potential attackers and target capabilities:


Information security stakeholders embed security controls to reinforce their defenses across 5 primary layers as identified above. The preventive and detective controls across each layer are governed through policies and procedures based on good practices and regulatory requirements. Is this enough for safeguarding our business environments with such security layers and control strategies? The logical answer lies in looking over to the other side of the fence; the state of security vulnerabilities!

Sorry State of Security
There are hundreds of millions of attacks executed over the Internet that use millions of compromised hosts; on top of that, there are millions of URLs that are still serving malicious content. This brings us to our previously discussed point: the spectrum of cyber threats is getting bigger and their nature is getting advanced and persistent day by day as Internet usage and connectivity further penetrates the globe.

According to Symantec Corporation’s Internet Security Threat Report 2014 (2), 552 million identities were breached in 2013. These breaches allowed attackers to gain access to credit card information, birth dates, government ID numbers, medical records, financial information, email addresses, login, passwords, and other personal information. Similarly, 1 in every 196 emails sent contained a virus.

Furthermore, the same report stated that 67% percent of websites used to distribute malware were legitimate websites that had been compromised by attackers. The risk also exists for mobile devices, as the report showed that 38 percent of mobile users had been exposed to mobile cyber crime.

Advanced attacks with specific motives were also witnessed during 2013 – targeted attacks and breaches against large US media outlets, US Federal Reserve site breach, biggest DDOS attack producing 300 gigabit/sec amount of traffic, leakage of 1.7 million classified NSA documents, Adobe breach impacting 38 million users and Target retail stores POS breach impacting more than 40 million users are some notable security incidents occurred in 2013.

This sad and sorry state of security affairs points to a missing link between the way we currently manage security and the way adversaries are successfully targeting our information assets. Let us further analyze this scenario in a more philosophical way!

It is about how prepared you are in detecting and recovering from an attack – data breach, small or big, is highly likely to happen in your business environment!

The Case for Intelligence driven Security
The security industry has been attempting to address the challenges of known-knowns (things we know that we know!) and known-unknowns (things we know that we don’t know!). This gave rise to security technologies based on signature/ patterns and correlation/ trend analysis respectively. Are we in a position to address today’s advanced threats fuelled by sophisticated blended techniques and availability of ever-increasing attack surface courtesy of big data? An honest reply – not really! This current challenge is characterized as a state of unknown-unknowns – an undetectable situation till intelligence is acquired to treat unknown risks to our business environments. The notion of intelligence driven security empowers business users and security stakeholders to tackle unknown situations in a more formidable manner.

Intelligence-driven security is yet to set its upper limit. There are numerous data sources for acquisition, further analysis and subsequent extraction of actionable intelligence through big data analysis techniques. The more we are able to correlate and establish context, the better will be the state of security intelligence to deal with sophisticated security threats lurking around our business environments. Here (Diagram 1) is an attempt to illustrate collection of diverse data from various sources.


Data from user, application, system and network levels from multitude of security and non-security devices, vulnerabilities and threat intelligence acquisition, feeds from configuration management, policy compliance, change management database and information gathered from risk management systems when fed into big data analytics engine provide a certain level of security intelligence to better understand business environments in view of security threats and to deal with various states of unknown-unknowns with better visibility. To better understand situations where big data analytics enables intelligence-driven security, let us examine data exfiltration/ data export scenario. (Diagram 2)


As security stakeholders, we need to ask some fundamental questions to make sure that data export was legitimate. These core queries may constitute:
1. How can we establish a trace route for a set of data being transmitted out from our network and ensure that data export is legitimate?
2. Can we identify business criticality of transmitted data and at the same time explore how vulnerable those servers and applications were when the data was exported?
To seek intelligence in order to explore required information:
– Outbound data monitoring to demonstrate that data leaving the organization was monitored
– Network packet capture to determine what content was exported.
– Data leakage prevention to identify why data was permitted to leave keeping in view criticality associated with it
– Risk management to guide further association of exported data with organization’s business critical services and what risks were evaluated with the same set of critical services
– Vulnerability management to identify the vulnerability state of infrastructure and applications on which that data was residing or being processed
The big data analytics engine processes the required information based on well defined correlation rules and provides visibility to steer through the situation with right information.

End-to-End Security Intelligence Management
Based on context set forth in view of constant rise of big data, sorry state of security, existing capabilities of security fraternity and logical need of having security intelligence, let us arrive at summing up the entire discussion with a refreshed view of end-to-end security intelligence management.

To deal with undetectable situations in our business environments, we tend to seek security intelligence systems powered by big data analytics to demonstrate attributes as briefed below (Diagram 3)


The capability of big events correlation coupled with historical data archive enables deep dive data inspection that creates an activity benchmark under some specific context. This propels the ability to detect anomalies. The intelligence system will demonstrate advanced monitoring capability based on activity benchmarks and flexible reporting mechanism to generate scheduled and on-the-fly reporting for further analysis and as feedback to management. All these attributes sum-up to develop situational awareness and context based security for making informed decisions and arriving at logical results. Last but not least, such systems are able to define patterns for future reuse and further refinement.

Intelligence-driven security allows organizations to effectively secure their most critical assets and manage cyber risk. The combination of big data and advanced analytics is the best way to defend an organization from sophisticated cyber attacks. The security industry has also realized its importance and many vendors are offering security tools with big data analytics capability.

Big data analytics offers great potential to a variety of industry verticals in terms of innovation, productivity and efficiency. This same potential applies to information security as the vulnerability state of affairs demands for “contextual intelligence” to deal with ever increasing advanced threats. It is fair to say that big data analytics propels security intelligence through collection, correlation and analysis of data from diverse sources. Organizations therefore need to adopt an intelligence driven security model that is risk centered, context based, continuously monitored and ready to respond. Based on this discussion, such an organizational readiness is very much needed to acquire the ability of finding a needle in a haystack!

1. us/en/ communications/internet-minuteinfographic .html
2. us /enterprise/other_resources/b-istr_main _report _v19_21291018.en-us.pdf


KAMRAN AHSAN, MASc, CCISO, CPISI is a Senior Director of Security Services at Etisalat’s Digital Services Business.