Security metrics can help measure how effective security efforts are. When you want to improve security Performance, you need to determine whether you are making progress. This is accomplished by using key performance indicators (KPIs) to help identify more specifically how security is performing. You also need to know if you’re taking risks. Key risk indicators (KRIs) are used to accomplish this.
Both KPIs and KRIs are part of the governance, risk, and compliance (GRC) efforts required to provide security oversight and tracking for an organization. While some metrics and measures are gathered manually, GRC tools are increasingly popular tools for IT and security risk management and oversight.
Key Performance Indicators
Key performance indicators are used to show how the organization is performing based on goals set by leadership, as well as progress toward the goals that have not yet been met. Measuring performance requires an understanding of what meaningful progress is and how it can be measured. Much like any metric, numbers themselves do not have meaning without context that is relevant to the organization and its objectives.
Major industry standards for measuring information security performance include the following:
- NIST SP800-55, “Performance Measurement Guide for Information Security” (https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800- pdf)
- ISO 27004, “Information technology – Security techniques – Information security management – Monitoring, measurement, analysis, and evaluation”
- ITIL’s information security management module
Several steps are necessary when identifying useful KPIs. For instance, what are the business goals? What data sources provide information regarding the goals? Can the sources of data be used to identify progress toward reaching those goals or indicate the current status in relationship to organization objectives? How frequently should the data be collected? Is the data collection automated or manual? By answering these questions, you’ll be better able to identify the KPIs that are useful to your organization. In addition, you’ll have a good idea of the task in front of you and the work necessary to accomplish it.
To define useful KPIs, several pieces of information are necessary. You first need to identify the baseline you are starting from and the goals you want to accomplish. Then you can best determine how to achieve those goals.
The baseline—where you are at—is the starting point for developing KPIs. It merely requires taking measurements of the current status. Of course, you must wisely choose which numbers matter. You want to choose factors that indicate the level of security. The KPIs selected should align with the organization’s documented goals. In addition, the more readily available the data or the easier it is to collect, the better. Automation is also a key component in being effective. If it takes an inordinate amount of time to gather the data, then the overall effort suffers.
Once the initial data is gathered, the baseline is created. At the next sampling inter- val, progress toward goals can be evaluated. Analysis of the data can begin. The analysis may be automated, manual, or both, depending on the circumstances. Additional information can also be considered during the analysis, such as external factors that may have influenced the data. For instance, an increase in unresolved calls to the help desk may have been created by issues with an application update. This in turn may indicate that testing for the application update was insufficient. Ensure that all the factors that impact the KPIs are documented. This allows for more accurate conclusions.
The information gathered should be documented. Trends should be observed and analyzed and findings reported to management as appropriate. With this data, determinations can be made regarding budget and resource allocation so that KPIs can show evidence of progress (or the lack thereof) toward organizational goals.
Key Risk Indicators
KPIs measure how well things are being done. KRIs, however, measure the organization’s risk and how its risk profile changes. This provides the ability to assess the likelihood of a negative event, as well as assess the risk level of an activity or situation. The risks, if realized, can typically profoundly impact the organization. Therefore, KRIs can strongly impact decision-making by senior-level executives.
Common key risk indicators include the following:
- Vulnerability metrics
- Policy exception rates
- Audit findings that are outstanding or unresolved
- Security incident rates
- Malware infection rates
- Security education and awareness rates
- Measures of the use of unapproved software
- Risk assessment ratings
- Patching compliance
- Account management status and issue rate
KRIs, like KPIs, must be adapted to the organization that will use them and should be both timely and a good fit with the organization’s security needs.
Risk also changes over time. The activities of an organization can increase or decrease risk. The resultant KRI should indicate these changes. Management should identify acceptable measurements for any KRI. Any changes in a KRI that result in an unacceptable value should trigger an action on the part of the company. For instance, if the number of unpatched machines with critical vulnerabilities exceeds an agreed-upon percentage, then desktop support staff works after hours to manually patch enough machines to bring the KRIs back into a state of compliance.
