Applied Security Visualization

rsiles writes "When security professionals are dealing with huge amounts of information (and who isn't nowadays?), correlation and filtering is not the easiest path (and sometimes enough) to discern what is going on. The in-depth analysis of security data and logs is a time-consuming exercise, and security visualization (SecViz) extensively helps to focus on the relevant data and reduces the amount of work required to reach to the same conclusions. It is mandatory to add the tools and techniques associated to SecViz to your arsenal, as they are basically taking advantage of the capabilities we have as humans to visualize (and at the same time analyze) data. A clear example is the insider threat and related incidents, where tons of data sources are available. The best sentence (unfortunately it is not an image ;) that describes SecViz comes from the author: 'A picture is worth a thousand log entries.'" Read on for the rest of rsiles's review.

Applied Security Visualization
author	Raffael Marty
pages	552
publisher	Addison-Wesley Professional
rating	9/10
reviewer	rsiles
ISBN	978-0321510105
summary	Definitely Security Visualization is one of the most relevant present and future topics in the security field, and this book is simply THE reference.

This is a great book that joins two separate worlds, visualization and information security (infosec). The first chapter is an excellent introduction to the human perception system, its basic principles, and how we analyze, discern, and assimilate information. It is an eye opener for those new to the field. Chapter two is similar from an infosec perspective, and summarizes the main challenges and data sources, such as packet captures, traffic flows, and firewall, IDS/IPS, system, and application logs. The third chapter details different graph properties and chart types, including some open-source and online tools for chart and color selection. Although we (infosec pros) are familiarized with link graphs to represent relationships between botnet members or hosts, the book provides a whole set of charts for different purposes; one of the most useful types, and we are not very used too it in the security field, is treemaps. The chapter includes a really useful table to select the right graph based on the purpose of the analysis and the data available.

Then, the previous chapters are smoothly mixed together through a reference methodology that defines what is the problem to solve, and the process to manipulate the available data and generate a (or set of) graph(s) that allow gathering relevant conclusions and answers. The methodology is complemented with an introduction to the standard Unix-based text processing tools (grep, awk, Perl, etc). This methodology is later on applied, with a strong hands-on and how-to spirit, to an extensive set of common security use-cases, such as the perimeter threat, compliance, and the insider threat.

The perimeter chapter offers a deep insight into common attack scenarios, such as worms, DoS or anomaly detection, and operational tasks, like firewall log and ruleset analysis, IDS tuning, or vulnerability assessments. I could never forget how useful were SecViz techniques for anomaly detection on a huge DNS-related incident I was involved about 5 years ago. Thanks to the performance and statistical graphs we had available at that time, we were able to easily identify and solve a very complex and critical security incident.

When I saw this chapter included a wireless section I got really excited due to personal interest. However, I was disappointed as it was just a couple of pages. I think it could be extended to gather a whole set of useful information about complex wireless attacks and client and access points relationships, just by inspecting the different 802.11 management, control, and data frames, and even radio-frequency signals (from a spectrum analyzer). SecViz opens the door to a whole new wireless research area!

The compliance chapter offers a whole methodology to check and manage regulations, control frameworks, auditing, and risk monitoring and management from a visual perspective.

The same applies to the insider threat chapter, as it provides an impressive framework, not only visualization-based, to deal with malicious insiders. It is based on setting up scores for certain behaviors and activities (precursors), generating lists of suspicious candidates, and apply thresholds to accommodate exceptions. It also contains an extensive and directly applicable precursor list at the end to detect suspicious insider activities.

Finally, the book contains a whole chapter, full of references and comparison tables, of open-source and commercial visualization tools and libraries that allow the reader to select the appropriate tool for specific tasks and scenarios.

Although the book hands-on component is very significant, with lots of detailed examples of commands, scripts, and tool options to generate the different graphs, I would have liked to see a thorough usage of the how-to portions, as for some sections there are no specific details about how the graphs have been generated. The book layout makes it the perfect candidate to become a fully interactive technical book. I would suggest to add (for a 2nd edition ;)) practical sections to each chapter where the reader could reproduce all the steps discussed. The book CD is the perfect tool to provide the reader with all the (sanitized) data sets and logs used to generate the graphs, and even allow to include some challenges where the reader needs to analyze the data and answer some questions after generating the appropriate graphs.

To sum up, this book is a mandatory reference for anyone involved in the operational side of infosec, doing intrusion detection, incident handling, forensic analysis, etc, and it can be applied to both, historical analysis and real-time monitoring. Additionally, I found it useful too for auditing and pen-testing professionals, as it provides great tips to generate relevant and efficient graphs for the associated reports.

The accompanying DAVIX Live CD is an excellent resource to start applying the techniques covered throughout the book through open-source tools, SecViz is the Web portal to expand your knowledge on this topic, and AfterGlow is (one of) the most relevant SecViz open-source tools.

You can purchase Applied Security Visualization from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Q1 Labs

[Anonymous Coward]

I used to work at a firm called Q1 Labs [q1labs.com]. Their founders quite brilliantly mapped aggregate network statistical information into visual presentations. In effect, the computer did what it does best: aggregate information. However their product offloaded the detection of anomalies in that aggregate information onto the human, and in particular it presented data in a way that the human brain's visual centre could readily observe patterns and deviations from those patterns (but which patterns and deviations are quite difficult for computers to detect naturally/automatically).

It's kind of offtopic, but I thought that was cool enough to share. :)

Missing the point

[Jay L]

I haven't read this book yet, but visualization tools ARE a significant part of pattern detection that we've mostly overlooked.

Much as we try to create smarter algorithms that can do feature extraction, clustering, etc., the best pattern-detection engine we have is still the human brain. There are very few systems that can detect patterns when we have NO idea what we're looking for; the brain comes pre-installed. Have you ever tried to do logfile analysis on a few thousand machines? Playing "management by exception" doesn't work at scale; even the rare errors show up a few times a second.

I saw a presentation a few weeks ago by Deb Roy, who's heading the Speechome [mit.edu] project at MIT. He's set up a bunch of cameras recording continuous audio and video in his house, in an attempt to map the language development of his son. That's a LOT of data to sift through - some 90,000 hours. Way too much for standard audio scrubbing/speedup, which would be the equivalent of our grep-a-log-file.

So they've had to develop some incredible visualization techniques that let you view higher-level patterns across multiple "rich data" streams - things like frequent patterns of motion (there's baby playing with his toy car with Daddy), eye-gaze focal points (there's baby looking at the car before saying "KA"), etc. that just pop out at you as you view the full data stream. It's truly jaw-dropping stuff, and it's applicable to far more than speech.

Anyone here ever defragged a hard drive (yeah, I know, ext3/HFS/etc.)? Would you get a better feel for the operation if you saw a list of sector numbers that were being relocated, or the usual 2D colored-block graph?

Anyone ever seen TreeMaps for finding large files on your drive?

Anyone ever known when a process is about to crash because the patterns of UI hesitation and hard-drive head-movement sounds change as the core files get written out?

That's all that info-vis is. It's presenting data in a way that lets you use intuition and subconscious cues to find what you're looking for - even if you don't KNOW what you're looking for.

Here's Deb Roy showing how you turn motion patterns from multiple video cameras into a two-dimensional, printable chart:

Visualization generation [mit.edu]