What is SIEM, and How Does It Improve Security?
July 12th, 2018 by admin
I recently came across a PowerPoint presentation entitled, “SIEM for Beginners: Everything You Wanted to Know, But Were Afraid to Ask.*” I want to share it here (in a re-formatted version), because it’s one of the best explanations I’ve come across on the subject. You don’t have to be a cybersecurity expert to grasp the concept of how it works and understand its tremendous power in providing a better defense against the assault of ever-growing, and perpetually changing, security threats.
Although the IT industry has settled on the acronym “SIEM” as the catch-all term for this type of security software, it evolved from several different (but complementary) technologies before it. These include:
- LMS – “Log Management System” – a system that collects and stores log files from multiple hosts and systems into a single location, allowing centralized access.
- SLM /SEM – “Security Log/Event Management” – a log management system focused on highlighting log entries as more significant to security than others.
- SIM – “Security Information Management” - an asset management system, but with features to incorporate security information too.
- SEC – “Security Event Correlation” – To a particular piece of software, three failed login attempts to the same user account from three different clients, are just three lines in their logfile. To an analyst, that is a peculiar sequence of events worthy of investigation. Log correlation (the process of looking for patterns in log files) is a way to raise alerts when these things happen.
SIEM, short for “Security Information and Event Management” is the term used for a solution that merges all of these technologies into a single product, and the generalized term for managing information generated from security controls and infrastructure.
Why you should care about your log files
The information needed to answer the questions, “Who’s attacking us today?” and “How did they get access to all of our organization’s information?” are found within log files.
A log file is a file (typically plain text) that keeps a record of events, processes, messages and communication between various software applications and an operating system. Logging is the process of capturing and storing log file entries. Some very basic examples of when log files are created are when you: sign in to or out of the company network, access a website, or download a file. Just the operation of your firewall alone, can generate 30-50 events per second! So, once you understand what a log file is, you also realize how quickly a mountain of them are created.
We may believe that the security controls we have in place (e.g. a firewall and/or antivirus) contain all the information we need to do security - but they often only contain the things they’ve been designed to detect. Meaning, there’s no “before and after the event‟ context generated within them. Having the right context is usually a vital part of separating a false positive (a misconfigured system) from true security event (someone is attacking my web server). These days, successful attacks on computer systems rarely look like real attacks - except in hindsight. If this weren’t the case, we could automate all security defenses without ever needing to employ human analysts. Attackers are no longer teenagers working from their basement trying to get into your network for kicks. Today’s attackers are typically highly-trained and highly-motivated cybercriminals who often try to remove and falsify log entries to cover their tracks. Which makes having a source of log information that can be trusted vital to any legal proceeding arising from computer misuse.
Looking at security through a wider lens
SIEM is about looking at what’s happening on your network through a larger lens than can be provided via any one security control or information source.
- Your intrusion detection system only understands packets, protocols and IP addresses
- Your endpoint security (antivirus, etc.) sees files, usernames and hosts
- Your service logs show user logins, service activity and configuration changes
- Your asset management system sees apps, business processes and owners
None of these by themselves, can tell you what is actually “happening to your business,” in terms of securing the continuity of your business processes – but all together, they can.
SIEM is, essentially, a management layer above your existing systems and security controls. It connects and unifies the information contained in your existing systems, allowing them to be analyzed and cross-referenced from a single interface. With that said, SIEM is only as useful as the information you put into it; the more valid the information depicting your network, systems and behavior the SIEM has, the more effective it will be in helping you make effective detections, analysis and response in your security operations. So, in a nutshell, it’s a perfect example of the “Garbage In, Garbage Out” principle of computing.
The power of correlation
The heart and soul of a SIEM is log collection – the more log sources that send logs to the SIEM, the more that can be accomplished with the SIEM. Logs on their own rarely contain the information needed to understand their contents within the context of your business. And security analysts have limited bandwidth to become familiar with every last system that your IT operation depends on. With only the logs, all an analyst sees is: “Connection from Host A to Host B.” Yet, to the administrator of that system, this becomes “Daily Activity Transfer from Point of Sales to Accounts Receivable.” The Analyst needs this information to make a reasoned assessment of any security alert involving this connection. Which means that the true value of logs is in their correlation, which reveals actionable information.
A good SIEM deployment includes logs and alerts PLUS knowledge:
LOGS AND ALERTS - WHERE THEY COME FROM:
Security Controls
- Firewalls
- Intrusion Detection
- Endpoint Security (Antivirus, etc.)
- Data Loss Prevention
- VPN Concentrators
- Web Filters
- Honeypots
Infrastructure
- Routers
- Switches
- Domain Controllers
- Wireless Access Points
- Application Servers
- Databases
- Intranet Applications
KNOWLEDGE - WHERE IT COMES FROM:
Infrastructure Information
- Configuration
- Locations
- Owners
- Network Maps
- Vulnerability Reports
- Software Inventory
Business Information
- Business Process Mappings
- Points of Contact
- Business Partner Information
In terms of security analysis and threat identification, the power of log correlation is immense. Correlation is the process of matching events from systems (hosts, network devices, security controls and anything else that sends logs to the SIEM). Events from different sources can be combined and compared against each other to identify patterns of behavior that are invisible to individual devices… They can also be matched against the information specific to your business. In short, correlation allows you to automate the detection of things that should not occur on your network.
The beauty of log correlation is best demonstrated in the example provided below. It is the difference between:
“14:10 7/4/20110 User BRoberts Successful Auth to 10.100.52.105 from 10.10.8.22”
and. . .
“An account belonging to marketing connected to an engineering system from an office desktop, on a day when nobody should be in the office.”
Creating actionable intelligence out of log files
As mentioned earlier, your network - even if it’s a small one - generates large amounts of log data. A Fortune 500 company’s infrastructure can generate 10 Terabytes of plain-text log data per month - without breaking a sweat. You can’t hire enough people to read every line of those logs looking for bad stuff (don’t even THINK about trying this). Even if you succeeded, they’d be so bored they’d never actually spot anything even if it was right in front of their face; which it would be. Log correlation lets you locate the interesting places in your logs – that’s where the analysts start investigating from…and they’re going to find pieces of information that lead to other pieces of information as the trail of evidence warms up. Being able to search through the rest of those logs for that one thing they suspect resides there, is one of the other key functions of a SIEM.
So, it’s a fair statement to say that a SIEM is fundamentally a giant database of logs. It would be amazingly useful if every operating system and every application in the world, recorded their log events in the same format – but they don’t. Most logs are written to be readable by humans, not computers. That makes using regular search tools over logs from different sources more difficult. Two log entries can say the same thing to a human being but are very different from the machine’s point of view. Long story short, to be most effective we’re going to need to break down every known log message out there, into some kind of normalized format. So, if you’re looking at information about how many devices a particular SIEM solution supports – what that really tells you is how many devices it can analyze the logs from.
Breaking those log entries down into their components, and normalizing them, is what allows us to search across logs from multiple devices, and correlate events between them. Once we’ve normalized logs into a database table, we can do database style searches, such as:
“Show [All Logs] From [All Devices] from the [last two weeks], where the [username] is [Broberts]”
This is also what allows us to do automated correlation - matching fields between log events, across time periods, across device types - that can establish “red flag” criteria, such as:
“If a single host fails to log in to three separate servers using the same credentials, within a 6-second time window, raise an alert”
Just as with any database, event normalization allows the creation of report summaries of our log information, such as:
“What User Accounts have accessed the highest number of distinct hosts in the last month?” or “What subnets generate the highest number of failed login attempts per day, averaged out over 6 months?”
So, we now can see that SIEM is a recording device for the systems that form your information infrastructure. SIEM allows you to give analysts access to information from these systems, without giving them access to the systems themselves. Event correlation allows you to encode security knowledge into automated searches across events and asset information to alert on things happening within your infrastructure. It effectively creates a starting point for human analysis by taking important information out of a sea of log data.
To better protect your organization against today’s highly sophisticated security threats, you need timely access to the relevant, actionable data that SIEM delivers. Along with our SIEM and Security Operations Center (SOC) resources, Advanced Network Systems provides a host of other cybersecurity services including internal and external vulnerability assessments, proactive security remediation and incident response. Our Managed Security Program takes the cost and complexity out of having a robust cybersecurity system, offering the most advanced services on the market at a price even small businesses can afford.
Posted in: Security, Cyber Security, Managed Services