How to Effectively Monitor Risk & Controls: Testing vs. Metrics

In today’s organizations, risk managers are tasked with the responsibility of effectively monitoring risk.  They need to know what to monitor and how to determine if mitigation activities are effectively preventing risks from materializing. Traditionally, organizations monitor activities through Control Testing, but this provides little more than a false sense of security for organizations.

A major weakness in just using Testing to monitor risk mitigation strategies and activities, is that testing usually tells you if an activity is internally being complied with, but not if the activity is actually adequately covering the risk or producing any business value.

In most organizations, controls are put in place to implicitly cover a risk, and soon after activities are put in place, everyone loses sight of the original purpose of the control in the first place.  It becomes an internal compliance activity, rather than a risk mitigation strategy.

A better way to monitor control effectiveness is through a formalized ERM process, where risks, mitigations, and monitoring activities are explicitly linked, and business metrics are leveraged to measure coverage through business results.

Collecting business metrics enables you to track the progress of your mitigation activities over time.  You can set targets and tolerance levels around these metrics causing warning signs to appear as metrics begin to move out of tolerance.  This allows you to take action before a negative outcome materializes.

Here’s an example of this theory based on a real customer’s situation.

A bank has an online banking system that goes down frequently and the subject matter expert on that system never seems to be available when there is an issue. The company then institutes a training program to cross-train more individuals. Often, organizations get caught up in testing the compliance or occurrence of the control, such as “Has every new IT hire completed the training within the first 6 months?” and lose sight of why the activity was implemented in the first place – in this case, to improve system uptime.

In this situation, once the bank began tracking the business metric of system uptime, they were able to see that there was no improvement from the control activity. The bank reinvestigated and realized that the system was going down during peak usage times, like lunch, when the subject matter expert was away from their desk.  The bank now can institute effective activities, like adding more memory to the system.

By tracking business metrics, organizations are able to more effectively mitigate existing risks and detect emerging risks long before they have significantly affected the organization.