Measuring a Cyber Awareness Culture

Measuring a Cyber Awareness Culture

Andreas Wuchner

Until recently, cyber awareness metrics have been treated by many as a tick-box exercise driven by regulations. The regulator requires x number of hours of cyber awareness training per employee per year, and once that is done, the organisation ticks a box and waits until next year. Regulators require phishing simulations, so you run a campaign, see who clicks and who doesn’t, and there’s your result. Done.

Should it be more important? Absolutely. If you look at the successful cyber-attacks over the last few years, about 90% can be traced to human error. Even if it was only 50%, that’s still really high. You have all these preventative cyber security tools in place, but they don’t stop everything. That’s why it is still essential to empower people to get this number down. Managing human cyber risk is a must!

More modern organizations are looking to improve. 

Success boils down to how much organizations really care about cybersecurity. Organisations typically believe the user is the biggest source of their security problems. Not many have thought to transform the human behind the computer into the strongest spearhead in their defensive strategy. Instead of viewing employees as liabilities, make them assets.

You need to educate everyone about cybersecurity but you must make it relevant for them.

Regulators and auditors are asking more frequently about organizations’ cybersecurity awareness and cultural programs. The sooner these organisations begin empowering their employees, the sooner they will improve their cybersecurity posture.

What are the Challenges?

Challenges can be summarized in two categories: technology and people. 

One of the problems in measuring cyber awareness culture is that data coming from tools like number of phishing clicks measures just clicks, not the culture itself. Ultimately, the tool only knows what it knows; it doesn’t give you data in context. Second, how trustworthy is the data? Is the tool/control deployed properly? Does it also cover mobile phones? Is coverage optimised? Is the data accurate and timely? Unless you have consistent, trustworthy data, you can’t make meaningful conclusions.

On the people side, it is really hard to build a cyber aware culture.

Buying a new security tool is much easier than making everyone in your organisation care deeply about cybersecurity. Many people just see a security tool as a hassle that gets in the way of them doing their job. You need to educate everyone about cybersecurity but you must make it relevant for them. Make them aware that cyber security is a whole-of-life issue. If you can make this the prevailing mindset, you have won half the battle. People care more when they see, and truly understand, how something abstract like cybersecurity can impact their or their families’ lives.

How Do You Overcome These Challenges?

A lot of successful cyber awareness culture programmes feature cyber champions – people in the organisation who really care about security and are willing to dedicate their time to improve the baseline security knowledge. They become advocates who talk about security, check up on their peers, and really drive a cybersecurity culture. This human element is much more effective than diktats from a faceless board.

Another solution is to gamify cyber awareness. You can have fun with a Wall of Shame for the team with the worst click rates, or, more seriously, use a Wall of Fame to celebrate teams that are outstanding in their cyber awareness. Acknowledge and celebrate those people or teams who practice good cyber hygiene. These are the people who are security assets rather than liabilities.

How Do You Measure Cyber Awareness Culture?

It can be very difficult to measure cyber awareness culture within an organisation. It is a matter of course that organisations will measure basic things like whether people have completed the cyber onboarding training, or how often people click on simulated phishing campaigns.

Measurement is often driven by regulatory requirements or frameworks. Indeed, the Security Controls Framework (SCF) has eleven controls around security awareness and training, and NIST groups them in the “protect” function. It is notable, however, that not all regulations require a security awareness element, such as ISO27001 and COBIT. This is perhaps a hangover from when cyber awareness was thought to be a “secondary” aspect of security.

Today, they ask ‘do you have a training programme?’ Eventually, they will ask ‘is your programme effective?’ This is a very different question and one not so easily answered.

You can measure these controls relatively easily, and thus prove to the regulator that your organisation is compliant, but this doesn’t provide you with any meaningful insight into your organisation’s cyber awareness culture. As a result, this tick-box attitude won’t help you reduce human cyber risk in the organization.

In the future, the auditor or regulator will change their line of questioning. Today, they ask “do you have a training programme”? Yep. “Has everyone done it”? Yep. Eventually, they will ask “is your programme effective”? This is a very different question and one not so easily answered.

Say I give you a test with ten questions about cyber awareness. What makes a strong password? How do you spot a phish? Things like that. At the end, I see that you got six out of ten questions correct. Six is the threshold, so you pass. But I don’t know if you actually know these things. Were you guessing? Did you get your assistant or some tech-savvy teenager you met at a café to take the test? 

In a modern organization that measures cyber awareness culture, you are asked the same questions, but you are measured in how long it takes to answer. You are also asked “how confident are you in your answers”? This insight reveals a lot more about you as a user and your organisation’s cybersecurity programme.

You can also learn much more about the security culture this way. An over-confident user who performs well on traditional cybersecurity tests may be more careful in the future. He may now hover over an email to see if it is in fact an urgent email from the CEO or a phish from a rogue Gmail account in North Korea. This is indicative of a bad cyber awareness culture.

An organization with a good cyber awareness culture has employees like these. These employees are assets. They have engaged with the training, they understand personal digital security, and can confidently answer all ten questions quickly and correctly.

Another good example of how to measure cyber awareness culture is the use of auto-locking screens. Many companies have a policy that when you leave a computer, you should manually lock it. And if you don’t, it will auto-lock after x minutes based on a group control.

A tool only sees numbers, so if you rely solely on that tool for your metrics, you’re getting a narrow view that lacks important context.

In an office environment, an unlocked screen may be harder to find as there are usually cyber-aware people around who could call someone out for this infraction. But so many more people are now in a home-office scenario, where there may be spouses, housemates, or children around seeing or using your device without appropriate training or permissions. Who knows what they might see or click on?

By tracking manual vs. auto-locks as a metric, you can get some understanding into cyber awareness culture. You can see that people have read the guidelines and done the training, so they know that policy dictates that they manually lock their screen when it’s unattended.

By contrast, if employees just leave their screens unlocked when they go out to lunch, and it runs into auto-lock, it says just as much about the organisation’s cyber awareness culture. In this case, the culture isn’t as good as it could be and/or there is a problem with the training. In either environment, policy needs to be meaningful – it shouldn’t just be “do this”, it should be about showing people why these things are important.

What Does Human Cyber Risk Monitoring Bring to Cyber Awareness Culture?

Awareness of the human risk in cybersecurity is just one element in understanding your overall security and risk posture.

The reason that human cyber risk monitoring is so valuable to security measurement is that it combines data from multiple disparate sources that wouldn’t normally interact. A tool only sees numbers, so if you rely solely on that tool for your metrics, you’re getting a narrow view that lacks important context.

For example, a tool may not know if there are new joiners or leavers in the organisation. Combining data from the awareness tool and an HR tool therefore provides an extra layer of context. Similarly, you may combine awareness data, with auto-lock data, and privileged access data. Then you can see if people who have access to critical business applications are also leaving their screen unattended before it goes into auto-lock. This data provides a lot more insight into your cyber awareness culture than whether someone passed an annual security training. 

By combining data in different ways, you can start to ask really interesting questions about cyber awareness culture.

If you have a lot of people with access to critical information who don’t care about cybersecurity, you need to know about it, because that means the securing training clearly isn’t resonating and there is a much higher risk of something going horribly wrong.

By combining data in different ways, you can start to ask really interesting questions about cyber awareness culture. Are all new joiners enrolled in mandatory training? Did they all get their first phishing test email in their first week? How often have they clicked, compared to people who have been with the organisation for ten years? Do younger employees click more or less than employees with longer tenures? Which departments click the most? Do managing directors click more or less than entry-level employees?

The answers to these kinds of questions provide much more meaningful insight than simply the number of clicks or auto-locks in isolation. With more insight, you can tailor your cybersecurity training or programme, which is much more effective than the traditional one-size-fits-all security test for your compliance checklist. With more insight, you can see and understand just how efficient and impactful your spend on cyber awareness really is at the end.

Consider this article a primer on cyber awareness culture and measurement. Cybsafe, a cyber security software and data analytics company that helps organisations manage their human cyber risk, has some excellent resources.

RELATED POSTS

Stagehand: Episode 4

Stagehand: Episode 4

Keith and I left the scene like we found it: the two kidnappers dead on the floor, their shotgun up against the wall, and the rope used to tie up Carl Timmons sprawled out on the floor. We tipped off local law enforcement and were gone before they arrived, leaving no...

Relationships in the Cyber Era

Relationships in the Cyber Era

The APT era is here. Attacks are becoming more common and the level of damage increasing in severity. As CISOs, we must prepare for the APT era. We must commit to changing our attitude and not adopting only advanced technological tools. The current awareness is not...

The Importance of Vendor Risk Management for CISOs

The Importance of Vendor Risk Management for CISOs

If a company deals with even one third-party vendor, then vendor risk management should be at the forefront of the CISO's mind. What is vendor risk management? Vendor risk management (VRM) is the process a company takes to verify that their suppliers and providers...

SecOps Needs More Democratization, Not Less SOC

SecOps Needs More Democratization, Not Less SOC

An increasing complexity of technologies, as well as an increasing number of failures and attacks followed by an increasing dependency on business goals is changing the way we run Security Operations Centers. I previously discussed the concept of a Fusion Center as an...

Good Enough Isn’t Good Enough Anymore

Good Enough Isn’t Good Enough Anymore

The cyber risks we face today are more than we faced previously but also fundamentally different in several respects. Our adversaries are more adept and their tools and tactics more protean in capability.  In light of these increasing challenges, our cyber defenses...

Stagehand: Episode 3

Stagehand: Episode 3

Cyprus ~ 2006 Ali Hassan was a low-level operative in Hezbollah, but we had it on solid authority that he knew where three high-level leaders of the terrorist organization were hiding. Keith arrived fifty-seven hours into Hassan’s interrogation and by the looks of it,...

Five Best Practices to do Supply Chain Security Right

Five Best Practices to do Supply Chain Security Right

Supply chain attacks aren’t new. In fact, The National Institute of Standards and Technology (NIST) published their initial report on supply chain risk back in 2015. One of the most well-known supply chain attacks happened shortly after in 2017. NotPetya corrupted...

Stagehand: Episode 2

Stagehand: Episode 2

Carl Timmons: CISO of Illuminating Solutions, a data analytics firm, forty-seven years old, never been married. Last Thursday, Carl arrived in San Jose on business. He was picked up by a company car and driven to The Manifeld Hotel. He was last seen leaving the hotel...

The Dark at the Top of the Stairs

The Dark at the Top of the Stairs

Let’s say you need to apply a critical patch across the organization, and the patch requires a reboot. While forcing a reboot to apply a critical patch is important, it creates business disruption that ripples out to your customers. Sooner or later, someone in the...

The Risk of Banking

The Risk of Banking

I just came off a big Zoom call with traditional bankers where they discussed changes in client behaviors, and the impact which new technologies bring, that fundamentally challenge today’s traditional European banking models. At the end of 2019, Boston...

Effective Board Communication for CISOs

Effective Board Communication for CISOs

Know Your Board If you’re a CISO, your Board generally knows who you are and what you do. But do you know who they are? No Board is monolithic. Each Board member brings unique value to the Board. Each is selected for what they add to the Board’s perspective, vision,...

Cyber Ops Must Evolve Towards Fusion Centres. Here is Why.

Cyber Ops Must Evolve Towards Fusion Centres. Here is Why.

Since the advent of space exploration in the 1960s, every child understands that the success of the space mission is dependent not only on the astronauts, but also on the engineers in the mission operation center. All complex missions or operations are high risk and...

Mitre Disrupting Advanced Persistent Threats
Share This