Security Information and Event Management (SIEM) tools have come a long way since their inception in 1997. The initial vision for SIEM tools was to be a ‘security single pane of glass,' eliminating alert fatigue, both in quantity and quality of alerts. Yet the question still remains: have SIEMs delivered on that promise, and if so, can every security team benefit from one? In this blog we'll dive a bit into the history behind SIEM products and their use-cases to give you context, regardless of if you're on your first SIEM purchase or looking to supplement your existing deployment.

First things first, what exactly is a SIEM?

Gartner describes a SIEM as “technology that supports threat detection and security incident response through the real-time collection and historical analysis of security events from a wide variety of event and contextual data sources.” Breaking that down, we at Rapid7 have found that customers are purchasing SIEM tools for three major use cases: log search, compliance, and incident detection and response.

For the first two purposes, SIEM tools are great when it comes to aggregating logs, searching across the dataset, and fulfilling auditor requests. SIEM tools also have the ability to correlate and analyze events across dissimilar sources, prioritizing the most important events. They also have real-time views, which makes it easier to spot trends, discovering ordinary patterns that could result in a breach. Another benefit? SIEM tools have reporting features that cover historical views of data collection, which can be helpful when detailing how your SIEM is performing. In our 2015 Incident Detection & Response Survey, we asked, "does your organization use a SIEM?" and the results we found are shown below:

Source: Rapid7 2015 Incident Detection & Response Survey

For modern incident detection and response, the tool leaves organizations wanting more.

Organizations report challenges with the high amount of alerts and false positives, the specialized skills required for effective use, and the growing deployment and maintenance costs. SIEMs are good at detecting anomalies, such as 10 failed login authentications, or the network IPs generating the highest firewall traffic. The challenge lies in differentiating anomalies from true malicious behavior, whether it be an attacker masking as an internal employee or an insider threat. For every SIEM alert, security teams need to answer numerous questions. Is this threat real, or a false-positive? What user had this IP address at this point and time? Who else is involved in the incident? Was there lateral movement beyond this incident to critical assets? In our 2015 Incident Detection & Response Survey we asked about the number of daily alerts teams receive from SIEMs, and compared that to the number of alerts a team could investigate in a day:

Source: Rapid7 2015 Incident Detection & Response Survey

Another challenge with SIEM tools is the specialized expertise required to use them to their full potential.

Put simply, detection rules are challenging to write and maintain. This is because SIEMs haven't been built to detect today's user behavior-based threats, which can make tuning the tool for detection and investigation feel unwieldy. This therefore requires expert security professionals to set up a successful SIEM. The time it takes to deploy a SIEM is also lengthy, taking at times months to fully set up before integration is complete. There's a high level of complexity that comes with hardware deployments, building detections, and integrations. SIEMs are a challenge to maintain, needing multiple team members to assure they are working properly. Dark Reading notes SIEM vendors “still have not done enough to simplify their products – particularly for small and mid-sized enterprises.” This puts companies at a disadvantage, as they generally do not have the same resources for SIEMs that large organizations do.

Perhaps most importantly, we come to the elephant in the room: cost.

When considering a SIEM, it's important to not only factor in initial purchasing costs, but also installation, hardware, and maintenance. While SIEMs can integrate with your existing security and network stack, tailoring detection to your network environment requires skilled individuals. A survey from elQnetworks reports that 52% of respondents require two or more full-time employees to manage their current SIEM deployment. It's important to note that there are two different components to managing a SIEM, one relating to software, the other to hardware. For software the question asked is, "is it running properly and detecting threats?" while for hardware the question is, "is my SIEM being fed and storing the right log files?"

Managing hardware for SIEM in particular presents two major challenges.

The first is employee time to keep the system running, and the second is that as log data expands, more and more hardware is needed. This is very resource consuming, and when combined with the fact that SIEMs do not scale particularly well, this can become a serious issue for organizations. In fact, the same elQnetworks survey revealed that 31% of respondents would consider replacing their existing SIEM solution for higher cost savings. This shows that while companies might enjoy the benefits of SIEM solutions, the cost side of the equation is adding discomfort and stress to the team.

Is there a better way?

As SC Magazine's eBook points out, SIEM tools have been struggling to keep up with demands of consumers, and as a result, “security analysts are turning to customized Big Data solutions.” At Rapid7, we've carefully identified these top pain points and are pleased to present our solution to solve your SIEM use-cases: InsightIDR. InsightIDR identifies attackers early in their infiltration, and shows you the exact users and assets involved without a mountain of alerts or tedious investigations. By providing immediate visibility across the network without having to wait for your security team to write complex rules, you can start getting from compromise to containment, fast.

Related Resource: Learn more about Incident Detection and Response with this complimentary toolkit.