Security Metrics 35
Ben Rothke writes "The goal of security metrics is to replace fear, uncertainty, and doubt
(FUD) with a more formalized and meaningful system of measurement. The FUD factor is the very foundation upon
which much of information security is built, and the outcome is decades of
meaningless statistics and racks of snake oil products. Let's hope that Andrew Jaquith succeeds, but
in doing so, he is getting in the way of many security hardware and software
vendors whose revenue streams are built on FUD." Read below for the rest of Ben's review.
| Security Metrics: Replacing Fear, Uncertainty, and Doubt | |
| author | Andrew Jaquith |
| pages | 336 |
| publisher | Addison-Wesley |
| rating | 10 |
| reviewer | Ben Rothke |
| ISBN | 0321349989 |
| summary | Authoritative text on information security metrics |
One could write a book on how FUD sells security products. One of the most memorable incidents was in 1992 when John McAfee created widespread panic about the impending Michelangelo virus. The media was all over him as he was selling solutions for the five million PCs worldwide he said would be affected. The end result is that the Michelangelo virus was a non-event. Nonetheless, it was far from the last time that FUD was used to sell security.
The allure of FUD is that companies can spend huge amounts of money fighting nebulous digital adversaries and feel good about it. They can then put all of that fancy hardware in dedicated racks in their data center, impressing the auditors with the flashing lights giving off an aroma of security and compliance.
And that is the chaos that security metrics comes to solve. Security metrics, if done right, can help transform a company from a nebulous perspective on security to an effective one based on formal security risk metrics.
Security Metrics is a fabulous book that should be in the hands of every security professional. The book demonstrates that companies must establish metrics based on their unique requirements, as opposed to simply basing their requirements on imprecise industry polls, best-practices and other ill-defined methods.
So why don't companies do that in the first place? If security metrics can provide even a quarter of the benefits that Jaquith states, companies should run to implement them. Real security metrics require an organization to open up their security hood and dig deep into the engine that runs their security infrastructure. It necessitates understanding the internal requirements, unique organizational risks, myriad strengths and weaknesses, and much more. Very few companies are willing to dedicate the time and resources for that, and would rather build their security infrastructure on thick layers of FUD. History has shown that the security appliance of the month almost always beats a formal risk and needs assessment.
Chapter 1 lays out the problem with approaches that most companies take to risk management. The main problem is that traditional risk management is far too dependant on identification and fixing, as opposed to quantification and triage based on value. Quantifying and valuing risk is much more difficult than simply identifying, since the software tools used do not have an organization context or knowledge of the specific business domain.
Chapter 2 sets out the foundation of security metrics. The goal of these metrics are to provide a framework in which organizations can quantify the likelihood of danger, estimate the extent of possible damage, understand the performance of their security organizations and weigh the costs of security safeguards against their expected effectiveness.
The time has come for security metrics since information security is one of the few management disciplines that have yet to submit itself to serious analytical scrutiny. The various chapters provide many different metrics that can be immediately used in most organizations to address that.
The author defines various criteria for what makes a good metric. One of his pet peeves is the use of the traffic light as a metaphor for compliance. Jaquith feels that traffic lights are not metrics at all, since they don't contain a unit of measure or are a numerical scale. He suggests using traffic lights colors sparingly, and only to supplement numerical data or draw attention to outliers. He astutely notes that if your data contains more precision than three simple gradations, why dilute their value by obscuring them with a traffic light.
The chapter concludes on what makes a bad metric, defined as any metric that relies too much on the judgment of a person. These metrics can't be relied on since the results can't be guaranteed to be the same from person to person. Also, security frameworks such as ISO-17799 should not be used for metrics. The book also tackles the sacred cow of risk management, namely ALE (annualized loss expectancy), and how it is significantly misused and misunderstood in the industry.
The book states that in developing metrics, there must be formal collaboration between the business units and the security staff. This collaboration serves to increase awareness and acceptance of security. In addition, it ensures that security requirements are incorporated into the lifecycle early on. This is needed as business units generally have no clue as to what the needed security requirements are.
Chapter 5 is a short course on analysis techniques and statistics. The author quotes George Colony who stated that "any idiot can tell you what something is. It is much harder to say what that thing means". With that, the book details a number of techniques for analyzing security data (average, median, time series, etc.) and how each one should be used.
Chapter 6 is about visualization and notes that most information security professionals have no real idea how to show security, both literally and figuratively. Part of the problem is that security is proliferated with esoteric terminology and concepts, and the lack of understanding risk management amongst the masses. Part of the reason for this difficulty in sharing the security message with management is that many security practitioners lack simple metaphors for communicating priorities. This is compounded by the fact that the message is often focused exclusively on technical security issues, as opposed to the underlying business issues, which is was management is concerned with. The chapter is invaluable as it weans one off the malevolent pie chart and traffic light PowerPoint presentation.
Marcus Ranum notes that people seem to want to treat computer security like its rocket science or black magic. In fact, computer security is nothing but attention to detail and good design. FUD is all about emphasizing the black magic aspect of hackers and other rogue threats. Metrics are all about the attention to detail that FUD lives to obfuscate.
Security Metrics: Replacing Fear, Uncertainty, and Doubt is one of the more important security books of the last few years. Jaquith turns much of the common security wisdom on its head, and the world will be a better place for it. Security metrics are a necessity whose time has come and this invaluable book shows how it can be done.
Ben Rothke, CISSP is a security consultant with BT INS and the author of Computer Security: 20 Things Every Employee Should Know.
You can purchase Security Metrics: Replacing Fear, Uncertainty, and Doubt from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Ive got the security tool you need (Score:5, Funny)
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Wireless communications, yes, but there's going to be cables supplying power if not externally, at least internally.
Snip snip: not just for babies anymore.
- RG>
Protection against WHAT? (Score:1)
ISO to the rescue (somewhere in the future) ? (Score:5, Informative)
About ISO 27004: "The scope is to "provide guidance on the specification and use of measurement techniques for providing assurance as regards the effectiveness of information security management systems. It is intended to be applicable to a wide range of organisations with a correspondingly wide range of information security management systems. [It] provides guidance for measurement procedures and techniques to determine the effectiveness of information security controls and information security processes applied in an ISMS. The purpose of the Information security management measurements development and implementation process, defined in this Standard is to create a base for each organization to collect, analyse, and communicate data related to ISMS processes. This data is ultimately to be used to base ISMS-related decisions and to improve implementation of an ISMS."
Also, NIST has also something to add to the issue of security metrics in SP800-55. A few links:
http://www.iso27001security.com/html/iso27004.htm
http://csrc.nist.gov/publications/nistpubs/800-55
Re:ISO to the rescue (somewhere in the future) ? (Score:5, Interesting)
A little example for you:
A few years ago, I used to work for a large UK company. My manager, conscious that information security is something which you have to take seriously these days, commissioned a consultant to come in and write up a security policy which would comply with the British Standard.
The consultant came in for a while, and in due course produced a document which was generally criticised for being extremely vague, but my manager brushed this off. As far as he was concerned, we needed a piece of paper which complied with the British Standard. This was such a piece of paper, so he was happy.
However, seeing as I was tasked with looking over the paper, I asked my manager a question. "This British Standard it complies with. Have you read the standard itself?"
No, he hadn't. He hadn't even looked into the cost of buying a copy of it, thinking it would be "too expensive" (yet he was prepared to pay a consultant to come in and write the policy - go figure). So I went to a library and had a look at a copy of the British Standard.
I'm paraphrasing here because this was a few years ago. But, broadly speaking, it said:
This was followed by several pages of very vague descriptions of the sorts of things which should be included - things like segregated access, password expiration and re-use policies and all sorts of other stuff besides - which, by an amazing and spooky coincidence, were practically identical to the policy which we'd paid the consultant for. If I didn't know any better, I'd say the consultant retyped the British Standard almost verbatim, leaving out the bit about how "it's not possible to write a generic standard".
Re: (Score:3, Informative)
Well, ISO 17799 (or more precisely ISO 17799:2005) is the "second generation" BS7799-1 (there's also BS-7799-2, which is the standard your organization might certify). The original ISO 17799:2000 (IIRC) was more or less the same as BS7799-1. ISO 17799:2005 has a lot of stuff rewritten, and also a
Re: (Score:2)
You're using the term FUD incorrectly (Score:5, Informative)
'Defined by Gene Amdahl after he left IBM to found his own company: "FUD is the fear, uncertainty, and doubt that IBM sales people instill in the minds of potential customers who might be considering [Amdahl] products." The idea, of course, was to persuade them to go with safe IBM gear rather than with competitors' equipment. This implicit coercion was traditionally accomplished by promising that Good Things would happen to people who stuck with IBM, but Dark Shadows loomed over the future of competitors' equipment or software. See IBM. After 1990 the term FUD was associated increasingly frequently with Microsoft, and has become generalized to refer to any kind of disinformation used as a competitive weapon.' From the Jargon file at http://www.catb.org/jargon/html/F/FUD.html [catb.org]
Re: (Score:1, Insightful)
Wouldn't this qualify as - and I quote your Jargon file - "disinformation used as a competitive weapon"?
How about you put the usage of FUD in context, rather than compusilvely belching some google'd historical data about the term for your karma's sake? Excuse me, but this is classic Slashdot crappy behavior.
Security companies are spreading FUD -
Re: (Score:2)
You're actually right, its not FUD (Score:1)
ISECOM's RAV and CIOview's SecurityNOW? (Score:1)
Of course, I'm biased being a volunteer at ISECOM, but I still think these are impor
bogosity quotient of ISECOM's RAV? (Score:3, Informative)
The supposed non-event (Score:3, Informative)
Coming soon to /. (Score:2)
Posted by FUD on Thursday, May 17, @07:12AM
Note from the author (Score:1, Informative)
Hi everybody,
First off, Ben -- thanks for the kind words. Glad to hear that you liked the book, and of course I am happy to see it slashdotted. :)
Second -- some of the comments have really focused on what the book ISN'T about -- fear, uncertainty, and doubt. Ben's done a good job talking about what the book IS about, but as the author I thought I'd add a quick comment about the goals of the book. Adapted from chapter one:
Goals of Security Metrics
The primary objective of this book is to quantitatively
Re: (Score:1)