Few things provoke more bitter arguments among security people than the question of whether compliance is security. I believe that broadly, compliance is not security. In this article, I explain why I hold this opinion. I believe that compliance could be security, but the reality of how companies operate compliance programs, and how auditors audit, often means that it’s not. I explore this reality below.

What are we even talking about?

When I observe (and participate in) these arguments, I sometimes see that part of the reason why people cannot agree is that they mean different things when they say “compliance is security” or “compliance is not security”. This makes the entire argument futile; person A is not supporting the thing person B is opposing. To avoid this futility, I will clarify what I mean by compliance being security.

Compliance being security means that people within a company, who want to know how secure the company is, can look to the state and outcomes of security compliance programs to answer their question.

This meaning stands in contrast to a few other meanings I’ve encountered:

• This is not about whether compliance frameworks can make you more secure. More is a comparative word – of course they can.
• This is not about whether stakeholders external to an organisation can find value in examining the company’s security compliance certificates. Again, the answer to that question is an easy yes.

Now let’s get into it.

The many approaches to implementing compliance

Compliance frameworks have good intentions. They want us to manage risk, fix vulnerabilities, respond to incidents, and so on. But human beings manage to do wildly different things in response to the same compliance requirements. Let’s consider three hypothetical companies’ responses to a compliance requirement. In real life, any company might look like any of these from one compliance cycle to the next, or may be a combination of all three of these to different degrees.

1. Company A fulfills the letter and the spirit of the compliance requirement. They read and think about what it means, what it’s meant to achieve, and set about implementing it faithfully to achieve that outcome.
2. Company B fulfills the bare minimum of the letter of the compliance requirement. You want risk management? Here’s a methodology document, grabbed off SANS’ resources portal with our name find-and-replaced in. Here’s a risk register spreadsheet. Done.
3. Company C doesn’t do much of anything, but will use a range of techniques, from social engineering to generating false records for their auditor, to get through their compliance audit.

All three companies will get compliance certificates that look identical, and therefore communicate similar levels of security among them. This is the first problem with compliance-focused security – that pertinent differences between the security levels of different entities are abstracted away, and an even external appearance is granted to all.

The many approaches to auditing compliance

Analogous things happen on the auditor side. Audit is necessary, on the surface, because verification and reporting by third parties is considered more trustworthy than self-attestations. But in practice, things do not always go smoothly for auditors either. There are a number of factors that impact what auditors deliver, and how they deliver those things. Let’s consider three hypothetical audit firms and their teams.

1. Audit firm A has a team of solid auditors. But the market is changing, and their partners make it clear to them that this is a well-paying customer who must be kept happy. They cloak the message in talk about excellent customer service, but the meaning is unmistakable. That influences how the auditors conduct the audit, how deep they go into some things, and how lightly they skim over others.
2. Audit firm B has a mixed team. They have some brilliant auditors, but they’re not very experienced, because the firm has recently been experiencing a high amount of staff turnover. They also have crazy time pressures, and for all the “business alignment” and “value adding” they want to do, there’s not enough time to go into as much depth as they would like.
3. Audit firm C has a team of auditors straight out of school who were thrown into the deep end on day 2 of their job. Enough said.

Again, in real life, any team of auditors might operate like any one (or combination) of these three, at any point in time. A similar problem to the implementation problem appears here as well. All three teams produce certificates, and important differences in the way the audit was performed are abstracted away.

That pesky thing called scope

Compliance work often starts with scoping, and ideally includes a periodic review of that scope.

Since compliance work is often taxing both in fact and in perception, scoping is how we reduce both the real and the perceived burden on everyone involved in ensuring compliance. The fewer systems have to be touched, dismantled, re-architected, reviewed, or completely replaced, the better for everyone.

But what happens to systems that fall out of that scope? In an environment where security is the compliance programs and nothing else, those systems may be left out of security efforts. But that does not mean that the company cannot incur losses through those system.

AKA risk.

I know this firsthand. I once became domain admin on a penetration test by compromising a server that had been “decommissioned” and was no longer in use or in scope for any of the compliance programs, but still had some very insecure software, and connections to the wider network.

So is compliance security?

Answer the following questions. You’re only answering to yourself, so you can be very honest:

1. What combination of the implementation specifics listed above were present in your organization, in your last compliance cycle? What did you do with the compliance requirements? On the spectrum of “I need to pass the audit” to “I’m going to truly do the work”, where do you fall?
2. How good, really, were your auditors? How much time did they have to perform the audit? How many other audit projects were they on at the same time, and how did that affect your own audit? How much prior knowledge of the technologies you use did they bring? How much time did they have to learn your implementations of those technologies? If you had some convoluted implementation of a technology that violated a compliance requirement, would they be able to tell? How much (and what kinds of) pressure were they under, from their own superiors? On the spectrum of “I need to finish this audit and move on to the next one” to “I’m going to perform a thorough audit that truly respects the requirements”, where would your auditor fall?
3. How far away from the standard’s requirements could you stray, before actually failing the audit? Do your auditors have the nerve to fail you, or write an unfavourable report, without worrying about the consequences?
4. What is the difference between the set of assets that are in scope for X or Y compliance program, and the set of assets through which an adverse event can impact your organization? Are both sets equal? Is there a small intersection? A large one? None at all?

Answers may vary from one compliance cycle to another, and will definitely vary from one company to another. Use the answers to decide what it means that you passed that audit, and what it doesn’t mean, and use those answers to come to your own conclusion. Is compliance security?