Most mechanical engineers are familiar with the three competing targets „time to market“, „costs“ and „quality“. And most of them also understand the costs of changes: the earlier a change happens in the product development cycle, the cheaper it gets.
An equivalent of these tripod for governmental organizations could be: time to completion, co-signing, collaboration & delegation and comprehensibility. And in security? The most obvious three axis are: Quality of detection, time to remediation and security efficiency.
Quality of detection is the first target. It covers the most urgent question: Do I really detect all attacks I need to detect without over flooding my crew with false positives? And this question leads to additional ones. Can I really detect all important attacks by supervising only the endpoint? How do I correlate information from the endpoint, the network and the datacenter? And how reliable and fast is my threat intelligence information for these three sensors?
Time to remediation is the second target. Once an incident is identified, how easy is it to drill down to patient zero, the first infection? How much time do you need to identify all other devices also attacked and penetrated? And how much time do you need to secure the rest of your system from those already infected.? Not to mention cleaning and resetting all compromised devices. And there is a similarity to engineering change here – the faster you fight an attack, the less impact this attack has on you overall system.
Security efficiency is the third target. How much resources do you need to implement a reliable holistic prevention system and to create the capacity to quickly react and defend an attack. And how much impact does your security solution have on your operational processes? Security measures which reduce the efficiency of your business processes may be cheap in security but will cost you a fortune in OPEX.
Measuring these three targets – or indicators, identifying KPIs is a difficult task, as one major component in analysis is unknown – the total number of attacks run on a specific system. So the number of attacks found by a specific security system could either be zero as the security system does not work or it could be zero as the target system does not undergo any attack. Therefore you need to define KPIs which work independently from the implementation or you need KPI which compare a before and after status.
An example for an implementation independent KPI is the ratio of false positive detections to the total number of incidents found. This is a clear KPI measuring the quality of a detection algorithm. On the opposite, the “number of incidents found“ needs to be compared in a „before and after“ manner, assuming that the total number of attacks run is constant over time.
Consequently, running intensive Proof of Concepts before selecting a new security solution is key to validate the efficiency of a solution over all three axes. I also believe that a holistic security architecture combined with an integrated portfolio of security components is best for delivering highest prevention and attack detection as well as fastest threat remediation while delivering the best operational efficiency.
An integrated solution detects attacks not only on a single point, but across the whole security architecture. Exchanging information between the security components and between the security, network and IT infrastructure increases the ease of investigation. And integrating IT and security operations improves the operational efficiency.
This Text appeared firs at linkedin.