Key Performance Indicators (KPIs) in security are painful. If you ask 3 different security engineers what they track and how they track performance, you’ll get 4 different answers. Ask for metrics on production application security and you’ll get 5, or more likely 0. With so much disagreement within the security community, it’s no wonder teams struggle to communicate priorities, expectations, and potential impact. If Facebook, Equifax, Under Armour, Panera Bread, and Google (the breach list goes on) have taught us anything, it’s that we can’t just measure our security effectiveness with a “Days Since Last Accident” sign.

Days Since Last Accident KPI

 

Poor metrics have a trickle up effect as well as trickle down. If AppSec or DevSecOps managers can’t communicate progress or value to the CISO, then the CISO will struggle to secure the right budget for the right tools. More importantly, the added benefit of reviewing a set list of AppSec KPIs is that it instills discipline within your org to make sure that what you need to pay attention to is actually paid attention to. For example, without the routine checklist of KPIs to monitor, you would otherwise miss any new servers or applications the engineering team rolled out and forgot to tell you about. Having a simple set of executive metrics that you can easily review each month will not only instill discipline to stay on top of the game, but also catch major security issues that would have gone unnoticed.

But rather than introduce more complexity to an already complex problem, we need to simplify. As Albert Einstein said “everything should be made as simple as possible, but not simpler.”

With that in mind, here are 6 easy to track application security KPIs that we use to track our effectiveness and areas of potential impact.

KPIs for Web Application Security


Weighted Risk Trend

Weighted Risk Trend (WRT) calculates the business risk from web application vulnerabilities over time. It factors in the impact of the application along with its risk as it pertains to the business. This metric is the one that everyone anguishes over. No one can agree on how to do track this “correctly”. Rather than arguing the nuts and bolts, we can all agree that nothing is secure and none of us have unlimited time and resources to secure all the things at all times.

So we like tracking this metric for 3 reasons:

  • Keep an honest record of what applications absolutely need immediate attention
  • Highlight which applications can be put into block mode until it can be prioritized
  • Show that progress is being made towards fixing the critical issues

 

Remediation Calculation Window

Now, every security professional knows what is patched and secured today may not be patched and secured tomorrow. So, knowing how quickly your team can detect vulnerable package information and create a patch is gold. Not only will improving this metric help keep your workload under control, but tracking the number of days since the vulnerability was detected to the time it was fixed will reduce the time an attacker has to access the application or the network.

 

Application Testing Coverage

This is the Donald Rumsfeld of AppSec metrics: Solving the unknown unknowns. Without a RASP tool in your pre-prod and prod environment, you just won’t have the visibility to solve for the unknown unknown vulns (btw, if you don’t, you should really talk to us). But if you do, this ensures that your pentests have tested your entire application, entirely, in its totality. Completely. For example, how many APIs did your marketing team expose in its recent campaign? Were those tested? What about that campaign from last year?

 

Mean Time to Respond

With GDPR in full force, this metric is far more important than it has been in the past. Setting up workflows and alerts to trigger within a 72 hour window is crucial to staying compliant. If you’re not in that window, figuring out what your benchmark is and working toward that goal is something you can easily track.

 

Confirmed exploits

This metric hurts, but we have to be accountable to it. If/when you do have an incident, you need to know about it. The upside of tracking this is that these are the issues that you can prioritize, take back to your team, and use to improve security training or your SDLC.

 

Confirmed Account Takeovers

Especially important for any financial services company, the ability to immediately identify compromised accounts will not only improve your response time, avoiding costly penalties, but will help aid in tracking of malicious activity within the account from wire transfer fraud to accessing other accounts.

 

While security KPIs can seem daunting and not worth the effort, we put together a simple workbook to help keep track of these KPIs, easily share it among your team, and help you keep an eye on the most important issues. You can download the spreadsheet here.

Enjoy!

 

Additional reading…

https://www.sans.org/reading-room/whitepapers/analyst/metrics-manage-application-security-program-36822

https://www.owasp.org/images/7/77/Magic_Numbers_-_5_KPIs_for_Measuring_WebAppSec_Program_Success_v3.2.pdf