From the perspective of our Product team, the challenges of risk and compliance professionals are at the forefront. This collaborative group of developers, designers, lawyers and risk managers uses those insights, along with rigorous R&D, to inform the way LogicManager works. It’s no question that they’ve got a lot to share, so we’ll be publishing these insights to our blog to help you make more informed business decisions.

In this blog post, our Product team dives into risk assurance: a critical data point in any risk management program. How much risk assurance can be automated, and how much subjectivity is required for it to still be helpful in controlling risk? Learn how to get the best of both worlds, and find out how ERM software like LogicManager can help.

Introduction

Risk Managers are asked to be prescient. When assessing a risk’s impact and likelihood, we’re trying to assess what would happen if the risk were to manifest in the future, and what’s the likelihood that would happen? While predictions of impact and likelihood can be informed by prior data, there’s still no replacing the assessment of subject matter experts, or the people that are facing the risk on a day-to-day basis.

But what about risk assurance, or the measurement of control effectiveness? Can we as risk managers automate the measurement of control effectiveness, and use that as an input in our calculations for residual risk?

The Case for Automated Assurance

The case for automated assurance is primarily that we know a lot of information about a control’s design and performance that can inform our understanding of whether it is operating effectively. We know whether a control is manual or automated, preventative or detective, and whether it’s a key control. Most risk managers should also have access to a control’s performance history, and whether it has passed or failed its latest testing. Moreover, KRI and KPI data can provide us real-time measurements of performance, such as system uptime, threats flagged or exceptions identified.

Each of these data points provides context, and it's natural for risk managers to use that context to try and calculate an assurance score automatically. We’re often approached by risk management teams that rate controls in the same way they rate vendors. For example, an automated control might score a 5 on the “effectiveness” scale, while a hybrid control would score a 2 and a manual control a zero. Frequently, these calculations get quite complex and account for multiple controls applied against the same risk. With tremendous amounts of data, is capturing a subjective assessment of control effectiveness still necessary?

The Case for Subjective Assessments

The case for subjective assessments is that calculated assurances don’t always translate to real-world scenarios. To take an extreme example, if the risk you’re managing is the threat of a meteor wiping out your data center, it doesn’t matter if you have 5 automated controls that are all passing their associated tests; there simply isn’t a way to fully mitigate every risk, and to assume otherwise — either by calculation or subjective assessment — is unrealistic. Finally, the assessment is further complicated when you consider that controls are not equally effective across all risks to which they apply. A control’s effectiveness must be assessed in the context of the risks it is mitigating, rather than in isolation.

Part of the move away from subjective assessments is the emergence of technology like artificial intelligence and robotic process automation, which can often account for many more variables and larger quantities of data than the human brain. But artificial intelligence is not a replacement for human judgment. While AI might better predict, for example, the chances of a certain control failure in the next year — whether it’s an all-hands-on-deck or business-as-usual failure — is better left to the risk manager.

The Best of Both Worlds

To most accurately automate the assessment of control effectiveness in your risk assessments, both subjective and automated measures should be employed. LogicManager allows risk managers to capture subjective assessments of control effectiveness from their risk owners, who are the subject matter experts in how well existing controls address their particular risks. These subjective assessments are then balanced by automated reviews and adjustments that account for real-time data collected about your controls.

This real-time adjustment is called the “Assurance Index,” and it’s LogicManager’s adjustment of a risk owner’s subjective assessment that accounts for the tests and metrics performed against a control. The Assurance Index allows risk managers to automate their assurance calculations, while still keeping the end result grounded in real-world scenarios. When a control fails a test or its metric falls out of tolerance, the Assurance Index will intelligently reduce the assessor’s control effectiveness rating for any associated risks. When the control is retested or its performance is improved, the reverse is true and the rating improves.

Risk managers can improve their programs by incorporating automation into their measurement of control effectiveness. The best programs can account for both control data and risk owner input to assess the confidence that risks are appropriately mitigated.