Select Page
The Risk of Measuring Risk

The Risk of Measuring Risk

Automated measuring of control effectiveness is a very good idea conceptually. When you can combine control gaps with relevant threat information, you get a very good picture about the actual technical cyber risks your business faces. If done correctly, it provides you with an end-to-end view of what is going on in your organization.

Unfortunately, organisations can’t confidently say their controls are really deployed everywhere they’re expected. As you know, hope is not a strategy. In many organizations, asset and service (as well as API) inventories are neither complete nor actual. Useful inventories require continuous triangulation and reconciliation between various data sources to assure organisations have accurate and complete control effectiveness.

The Road to Risk Mitigation: Measuring Security Control Coverage

This is why measuring risks and relying on their results only makes sense if you have a firm grasp on your security control coverage. Otherwise, you make decisions on faulty risk information. The security controls coverage metric lets you see just how broadly your controls have been deployed across your environment. This visibility is essential to the success of your overall cyber risk measurement programme.

The only way you can have true confidence in your overall security programme is to measure not only the operating effectiveness of your controls, but also measure the coverage of your controls. As a security professional, I want and need to know where I have gaps. It’s the things you don’t know about, that get you into trouble.

Compromises typically occur in the absence of a control, or when a control has failed. We all live in a highly dynamic world and the ongoing digital transformation continues to disrupt the status quo. These changes can also disrupt your controls; some may not deploy, some may be removed, or some may fail. Every security organization must be able to capture these deficiencies as soon as possible.

As a security professional, I want and need to know where I have gaps. It’s the things you don’t know about that get you into trouble.

A proper look at control coverage can deliver even more value. Controls coverage is an essential data point in risk quantification. Methodologies like FAIR (Factual Analysis of Information Risk) and Cyber VaR (Cyber Value at Risk) allow organisations to quantify risk. CyberVaR, in particular is very data-driven. It looks at a wide variety of aspects of security, risk and controls, including external threat landscape, internal events, threat scenarios, security capability, security controls coverage, and your overall security posture. It brings all of these together to give you a view of overall residual risk which can then be quantified into a value that’s meaningful to the business.

In order to provide a high level of confidence in your overall security posture, you need to know:

  1. your controls are working effectively, and
  2. you have 100% coverage, defined by your policies.

You must understand where your controls gaps are in order to address and remediate those gaps. If you don’t know where the gaps are, that’s where the compromises are most likely going to happen.

Your Key to Success: Automation

The route to success here is automation. When a process is automated, you get accurate results time and time again. You don’t have to question the data or the validity of the results.

Automation also lets you reduce your operational costs. Like it or not, every security function must find ways to reduce their operational costs and maximize their productivity. When you automate the processes around not just your controls coverage metrics, but all your security measurements, you reduce your cost of operations and scale faster. Industry benchmark studies show security teams often spend 36% of their time on reporting. Automating this process allows security people to focus more doingsecurity rather than reporting it.

When you automate the processes around not just your controls coverage metrics, but all your security measurements, you reduce your cost of operations and scale faster.

If automation is not an option, you will suffer the fate of creating quality controls coverage metrics manually. You will have to go to each tool individually, compile all the data together, then clean, aggregate, normalise, deduplicate and correlate all that disparate data. And by the time you’ve done all that and you’re ready to use the data, it might be out of date already. As a result, questions will arise around data integrity, and discussions about reducing risk devolve.

Don’t Forget to Communicate Your Controls

There are numerous stakeholders within an organisation who need to see your security metrics all the way up to board level. This applies to your controls coverage in particular.

The primary audience is the control owners, whether they are within the security function, infrastructure team, application development, or front-line staff. It’s important for the control owner to understand control coverage as well as how those controls are performing so that they can address any deficiencies or exposure. This is especially important for the front-line team, because they are responsible for managing the risk and they need to take action to address any gaps.

Some other stakeholder audiences are people in the compliance, audit and regulatory functions. These stakeholders must be able to rely on the controls data in order to make informed decisions, measure compliance to policy, and identify any gaps or risks within that environment. With complete, accurate data, these people can drive risk-based conversations and take actions as needed.

To wrap up, a common theme emerges here: trust in the data. When we all use the same set of data, we understand where and how it was derived and we have a high confidence in the data’s accuracy because it’s been automated. When everyone uses the same data set, and trusts it, discussions focus on risk and the right trade-offs and prioritisations, not about the accuracy of the reporting.

RELATED POSTS

Selling to a CISO? Practice Empathy, Not Salesmanship

Selling to a CISO? Practice Empathy, Not Salesmanship

The cyber security marketplace is hot. Ask any candidate for a cybersecurity role. Better yet, ask any supplier to CISOs. The supplier audience is especially vast, and it’s continuing to grow. Just three years ago, there were estimated to be less than 2,000...

Stagehand: Episode 4

Stagehand: Episode 4

Keith and I left the scene like we found it: the two kidnappers dead on the floor, their shotgun up against the wall, and the rope used to tie up Carl Timmons sprawled out on the floor. We tipped off local law enforcement and were gone before they arrived, leaving no...

SecOps Needs More Democratization, Not Less SOC

SecOps Needs More Democratization, Not Less SOC

An increasing complexity of technologies, as well as an increasing number of failures and attacks followed by an increasing dependency on business goals is changing the way we run Security Operations Centers. I previously discussed the concept of a Fusion Center as an...

Measuring a Cyber Awareness Culture

Measuring a Cyber Awareness Culture

Until recently, cyber awareness metrics have been treated by many as a tick-box exercise driven by regulations. The regulator requires x number of hours of cyber awareness training per employee per year, and once that is done, the organisation ticks a box and waits...

Good Enough Isn’t Good Enough Anymore

Good Enough Isn’t Good Enough Anymore

The cyber risks we face today are more than we faced previously but also fundamentally different in several respects. Our adversaries are more adept and their tools and tactics more protean in capability.  In light of these increasing challenges, our cyber defenses...

Stagehand: Episode 3

Stagehand: Episode 3

Cyprus ~ 2006 Ali Hassan was a low-level operative in Hezbollah, but we had it on solid authority that he knew where three high-level leaders of the terrorist organization were hiding. Keith arrived fifty-seven hours into Hassan’s interrogation and by the looks of it,...

Five Best Practices to do Supply Chain Security Right

Five Best Practices to do Supply Chain Security Right

Supply chain attacks aren’t new. In fact, The National Institute of Standards and Technology (NIST) published their initial report on supply chain risk back in 2015. One of the most well-known supply chain attacks happened shortly after in 2017. NotPetya corrupted...

Stagehand: Episode 2

Stagehand: Episode 2

Carl Timmons: CISO of Illuminating Solutions, a data analytics firm, forty-seven years old, never been married. Last Thursday, Carl arrived in San Jose on business. He was picked up by a company car and driven to The Manifeld Hotel. He was last seen leaving the hotel...

The Dark at the Top of the Stairs

The Dark at the Top of the Stairs

Let’s say you need to apply a critical patch across the organization, and the patch requires a reboot. While forcing a reboot to apply a critical patch is important, it creates business disruption that ripples out to your customers. Sooner or later, someone in the...

The Risk of Banking

The Risk of Banking

I just came off a big Zoom call with traditional bankers where they discussed changes in client behaviors, and the impact which new technologies bring, that fundamentally challenge today’s traditional European banking models. At the end of 2019, Boston...

Effective Board Communication for CISOs

Effective Board Communication for CISOs

Know Your Board If you’re a CISO, your Board generally knows who you are and what you do. But do you know who they are? No Board is monolithic. Each Board member brings unique value to the Board. Each is selected for what they add to the Board’s perspective, vision,...

Cyber Ops Must Evolve Towards Fusion Centres. Here is Why.

Cyber Ops Must Evolve Towards Fusion Centres. Here is Why.

Since the advent of space exploration in the 1960s, every child understands that the success of the space mission is dependent not only on the astronauts, but also on the engineers in the mission operation center. All complex missions or operations are high risk and...

Mitre Disrupting Advanced Persistent Threats
Share This