Select Page
The Risk of Measuring Risk

The Risk of Measuring Risk

Andreas Wuchner

Automated measuring of control effectiveness is a very good idea conceptually. When you can combine control gaps with relevant threat information, you get a very good picture about the actual technical cyber risks your business faces. If done correctly, it provides you with an end-to-end view of what is going on in your organization.

Unfortunately, organisations can’t confidently say their controls are really deployed everywhere they’re expected. As you know, hope is not a strategy. In many organizations, asset and service (as well as API) inventories are neither complete nor actual. Useful inventories require continuous triangulation and reconciliation between various data sources to assure organisations have accurate and complete control effectiveness.

The Road to Risk Mitigation: Measuring Security Control Coverage

This is why measuring risks and relying on their results only makes sense if you have a firm grasp on your security control coverage. Otherwise, you make decisions on faulty risk information. The security controls coverage metric lets you see just how broadly your controls have been deployed across your environment. This visibility is essential to the success of your overall cyber risk measurement programme.

The only way you can have true confidence in your overall security programme is to measure not only the operating effectiveness of your controls, but also measure the coverage of your controls. As a security professional, I want and need to know where I have gaps. It’s the things you don’t know about, that get you into trouble.

Compromises typically occur in the absence of a control, or when a control has failed. We all live in a highly dynamic world and the ongoing digital transformation continues to disrupt the status quo. These changes can also disrupt your controls; some may not deploy, some may be removed, or some may fail. Every security organization must be able to capture these deficiencies as soon as possible.

As a security professional, I want and need to know where I have gaps. It’s the things you don’t know about that get you into trouble.

A proper look at control coverage can deliver even more value. Controls coverage is an essential data point in risk quantification. Methodologies like FAIR (Factual Analysis of Information Risk) and Cyber VaR (Cyber Value at Risk) allow organisations to quantify risk. CyberVaR, in particular is very data-driven. It looks at a wide variety of aspects of security, risk and controls, including external threat landscape, internal events, threat scenarios, security capability, security controls coverage, and your overall security posture. It brings all of these together to give you a view of overall residual risk which can then be quantified into a value that’s meaningful to the business.

In order to provide a high level of confidence in your overall security posture, you need to know:

  1. your controls are working effectively, and
  2. you have 100% coverage, defined by your policies.

You must understand where your controls gaps are in order to address and remediate those gaps. If you don’t know where the gaps are, that’s where the compromises are most likely going to happen.

Your Key to Success: Automation

The route to success here is automation. When a process is automated, you get accurate results time and time again. You don’t have to question the data or the validity of the results.

Automation also lets you reduce your operational costs. Like it or not, every security function must find ways to reduce their operational costs and maximize their productivity. When you automate the processes around not just your controls coverage metrics, but all your security measurements, you reduce your cost of operations and scale faster. Industry benchmark studies show security teams often spend 36% of their time on reporting. Automating this process allows security people to focus more doingsecurity rather than reporting it.

When you automate the processes around not just your controls coverage metrics, but all your security measurements, you reduce your cost of operations and scale faster.

If automation is not an option, you will suffer the fate of creating quality controls coverage metrics manually. You will have to go to each tool individually, compile all the data together, then clean, aggregate, normalise, deduplicate and correlate all that disparate data. And by the time you’ve done all that and you’re ready to use the data, it might be out of date already. As a result, questions will arise around data integrity, and discussions about reducing risk devolve.

Don’t Forget to Communicate Your Controls

There are numerous stakeholders within an organisation who need to see your security metrics all the way up to board level. This applies to your controls coverage in particular.

The primary audience is the control owners, whether they are within the security function, infrastructure team, application development, or front-line staff. It’s important for the control owner to understand control coverage as well as how those controls are performing so that they can address any deficiencies or exposure. This is especially important for the front-line team, because they are responsible for managing the risk and they need to take action to address any gaps.

Some other stakeholder audiences are people in the compliance, audit and regulatory functions. These stakeholders must be able to rely on the controls data in order to make informed decisions, measure compliance to policy, and identify any gaps or risks within that environment. With complete, accurate data, these people can drive risk-based conversations and take actions as needed.

To wrap up, a common theme emerges here: trust in the data. When we all use the same set of data, we understand where and how it was derived and we have a high confidence in the data’s accuracy because it’s been automated. When everyone uses the same data set, and trusts it, discussions focus on risk and the right trade-offs and prioritisations, not about the accuracy of the reporting.

RELATED POSTS

Stagehand: S1 Episode 8

Stagehand: S1 Episode 8

Carl Timmons was given 24 hours to decide what he wanted to do. This was a tactic. Twenty four hours to sit alone and think about all the money he could want and the price he’d pay for it. And 24 hours to also contemplate what Andre Savin might do to him before...

Stagehand: S1 Episode 7

Stagehand: S1 Episode 7

Andre Savin and Lincoln Palmer had met on several occasions and had the type of relationship you’d expect between two men of their standings on the billionaire scale. Contemptuous but also understanding. They were both driven by the same desire—access to...

Stagehand: S1 Episode 6

Stagehand: S1 Episode 6

Belfast, New York - 1889 They called him The Boston Strong Boy—arguably the first real boxing star and one of the highest paid athletes of his time.  He’d always been good at school. He attended Boston College where his parents thought he might pursue...

What Is Zero Trust Anyway?

What Is Zero Trust Anyway?

About three minutes into planning this post, I had one of those “god, I am old” moments. Here is why I had the moment. I have worked in cybersecurity since 1994. My first job was at a big 3 working for the U.S. government through one of the world’s...

Stagehand: S1 Episode 5

Stagehand: S1 Episode 5

Kuwait, 1990 I’m launched out of a submarine a few miles off the coast of Kuwait City. When I swim to shore, I quickly change into my dry land clothes—a full burka. I was a six-foot-one Marine posing as a good Muslim woman. The catch, beneath the modest...

Ransomware: When Policy Matters Most

Ransomware: When Policy Matters Most

Most CISOs divide their approach to cyber defense into three pillars: people, technology, and processes. These pillars define a cybersecurity program’s defensive architecture and arsenal, available assets, and policies and procedures that together inform...

Selling to a CISO? Practice Empathy, Not Salesmanship

Selling to a CISO? Practice Empathy, Not Salesmanship

The cyber security marketplace is hot. Ask any candidate for a cybersecurity role. Better yet, ask any supplier to CISOs. The supplier audience is especially vast, and it’s continuing to grow. Just three years ago, there were estimated to be less than 2,000...

Stagehand: S1 Episode 4

Stagehand: S1 Episode 4

Keith and I left the scene like we found it: the two kidnappers dead on the floor, their shotgun up against the wall, and the rope used to tie up Carl Timmons sprawled out on the floor. We tipped off local law enforcement and were gone before they arrived, leaving no...

SecOps Needs More Democratization, Not Less SOC

SecOps Needs More Democratization, Not Less SOC

An increasing complexity of technologies, as well as an increasing number of failures and attacks followed by an increasing dependency on business goals is changing the way we run Security Operations Centers. I previously discussed the concept of a Fusion Center as an...

Measuring a Cyber Awareness Culture

Measuring a Cyber Awareness Culture

Until recently, cyber awareness metrics have been treated by many as a tick-box exercise driven by regulations. The regulator requires x number of hours of cyber awareness training per employee per year, and once that is done, the organisation ticks a box and waits...

Good Enough Isn’t Good Enough Anymore

Good Enough Isn’t Good Enough Anymore

The cyber risks we face today are more than we faced previously but also fundamentally different in several respects. Our adversaries are more adept and their tools and tactics more protean in capability.  In light of these increasing challenges, our cyber defenses...

Stagehand: S1 Episode 3

Stagehand: S1 Episode 3

Cyprus ~ 2006 Ali Hassan was a low-level operative in Hezbollah, but we had it on solid authority that he knew where three high-level leaders of the terrorist organization were hiding. Keith arrived fifty-seven hours into Hassan’s interrogation and by the looks of it,...

Mitre Disrupting Advanced Persistent Threats
Share This