Applied Security Visualization 45
rsiles writes "When security professionals are dealing with huge amounts of information (and who isn't nowadays?), correlation and filtering is not the easiest path (and sometimes enough) to discern what is going on. The in-depth analysis of security data and logs is a time-consuming exercise, and security visualization (SecViz) extensively helps to focus on the relevant data and reduces the amount of work required to reach to the same conclusions. It is mandatory to add the tools and techniques associated to SecViz to your arsenal, as they are basically taking advantage of the capabilities we have as humans to visualize (and at the same time analyze) data. A clear example is the insider threat and related incidents, where tons of data sources are available. The best sentence (unfortunately it is not an image ;) that describes SecViz comes from the author: 'A picture is worth a thousand log entries.'" Read on for the rest of rsiles's review.
This is a great book that joins two separate worlds, visualization and information security (infosec). The first chapter is an excellent introduction to the human perception system, its basic principles, and how we analyze, discern, and assimilate information. It is an eye opener for those new to the field. Chapter two is similar from an infosec perspective, and summarizes the main challenges and data sources, such as packet captures, traffic flows, and firewall, IDS/IPS, system, and application logs. The third chapter details different graph properties and chart types, including some open-source and online tools for chart and color selection. Although we (infosec pros) are familiarized with link graphs to represent relationships between botnet members or hosts, the book provides a whole set of charts for different purposes; one of the most useful types, and we are not very used too it in the security field, is treemaps. The chapter includes a really useful table to select the right graph based on the purpose of the analysis and the data available.
| Applied Security Visualization | |
| author | Raffael Marty |
| pages | 552 |
| publisher | Addison-Wesley Professional |
| rating | 9/10 |
| reviewer | rsiles |
| ISBN | 978-0321510105 |
| summary | Definitely Security Visualization is one of the most relevant present and future topics in the security field, and this book is simply THE reference. |
Then, the previous chapters are smoothly mixed together through a reference methodology that defines what is the problem to solve, and the process to manipulate the available data and generate a (or set of) graph(s) that allow gathering relevant conclusions and answers. The methodology is complemented with an introduction to the standard Unix-based text processing tools (grep, awk, Perl, etc). This methodology is later on applied, with a strong hands-on and how-to spirit, to an extensive set of common security use-cases, such as the perimeter threat, compliance, and the insider threat.
The perimeter chapter offers a deep insight into common attack scenarios, such as worms, DoS or anomaly detection, and operational tasks, like firewall log and ruleset analysis, IDS tuning, or vulnerability assessments. I could never forget how useful were SecViz techniques for anomaly detection on a huge DNS-related incident I was involved about 5 years ago. Thanks to the performance and statistical graphs we had available at that time, we were able to easily identify and solve a very complex and critical security incident.
When I saw this chapter included a wireless section I got really excited due to personal interest. However, I was disappointed as it was just a couple of pages. I think it could be extended to gather a whole set of useful information about complex wireless attacks and client and access points relationships, just by inspecting the different 802.11 management, control, and data frames, and even radio-frequency signals (from a spectrum analyzer). SecViz opens the door to a whole new wireless research area!
The compliance chapter offers a whole methodology to check and manage regulations, control frameworks, auditing, and risk monitoring and management from a visual perspective.
The same applies to the insider threat chapter, as it provides an impressive framework, not only visualization-based, to deal with malicious insiders. It is based on setting up scores for certain behaviors and activities (precursors), generating lists of suspicious candidates, and apply thresholds to accommodate exceptions. It also contains an extensive and directly applicable precursor list at the end to detect suspicious insider activities.
Finally, the book contains a whole chapter, full of references and comparison tables, of open-source and commercial visualization tools and libraries that allow the reader to select the appropriate tool for specific tasks and scenarios.
Although the book hands-on component is very significant, with lots of detailed examples of commands, scripts, and tool options to generate the different graphs, I would have liked to see a thorough usage of the how-to portions, as for some sections there are no specific details about how the graphs have been generated. The book layout makes it the perfect candidate to become a fully interactive technical book. I would suggest to add (for a 2nd edition ;)) practical sections to each chapter where the reader could reproduce all the steps discussed. The book CD is the perfect tool to provide the reader with all the (sanitized) data sets and logs used to generate the graphs, and even allow to include some challenges where the reader needs to analyze the data and answer some questions after generating the appropriate graphs.
To sum up, this book is a mandatory reference for anyone involved in the operational side of infosec, doing intrusion detection, incident handling, forensic analysis, etc, and it can be applied to both, historical analysis and real-time monitoring. Additionally, I found it useful too for auditing and pen-testing professionals, as it provides great tips to generate relevant and efficient graphs for the associated reports.
The accompanying DAVIX Live CD is an excellent resource to start applying the techniques covered throughout the book through open-source tools, SecViz is the Web portal to expand your knowledge on this topic, and AfterGlow is (one of) the most relevant SecViz open-source tools.
You can purchase Applied Security Visualization from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Re: (Score:1)
I apologize for the shortness of digital temper, I just quit smoking.
May I suggest restarting? I did a double take on "mandatory" but if you flash into dreams of turning things into hamburger after that something is wrong.
Re: (Score:1)
I did a double take on "mandatory" but if you flash into dreams of turning things into hamburger after that something is wrong.
At what point does the continual erosion of our rights, the continual battery which smashes the individual into a robot of compliance, how low must we sink, before we can revolt?
A picture is worth a thousand log entries? (Score:3, Insightful)
I think not. Unless it is a picture of those log entries.
Missing the point (Score:5, Interesting)
I haven't read this book yet, but visualization tools ARE a significant part of pattern detection that we've mostly overlooked.
Much as we try to create smarter algorithms that can do feature extraction, clustering, etc., the best pattern-detection engine we have is still the human brain. There are very few systems that can detect patterns when we have NO idea what we're looking for; the brain comes pre-installed. Have you ever tried to do logfile analysis on a few thousand machines? Playing "management by exception" doesn't work at scale; even the rare errors show up a few times a second.
I saw a presentation a few weeks ago by Deb Roy, who's heading the Speechome [mit.edu] project at MIT. He's set up a bunch of cameras recording continuous audio and video in his house, in an attempt to map the language development of his son. That's a LOT of data to sift through - some 90,000 hours. Way too much for standard audio scrubbing/speedup, which would be the equivalent of our grep-a-log-file.
So they've had to develop some incredible visualization techniques that let you view higher-level patterns across multiple "rich data" streams - things like frequent patterns of motion (there's baby playing with his toy car with Daddy), eye-gaze focal points (there's baby looking at the car before saying "KA"), etc. that just pop out at you as you view the full data stream. It's truly jaw-dropping stuff, and it's applicable to far more than speech.
Anyone here ever defragged a hard drive (yeah, I know, ext3/HFS/etc.)? Would you get a better feel for the operation if you saw a list of sector numbers that were being relocated, or the usual 2D colored-block graph?
Anyone ever seen TreeMaps for finding large files on your drive?
Anyone ever known when a process is about to crash because the patterns of UI hesitation and hard-drive head-movement sounds change as the core files get written out?
That's all that info-vis is. It's presenting data in a way that lets you use intuition and subconscious cues to find what you're looking for - even if you don't KNOW what you're looking for.
Here's Deb Roy showing how you turn motion patterns from multiple video cameras into a two-dimensional, printable chart:
Visualization generation [mit.edu]
Re: (Score:2, Funny)
[...] (there's baby looking at the car before saying "KA") [...]
Give the kid some credit; that's not "KA," it's "Car," with a perfectly enunciated Boston accent.
Re: (Score:1)
Re: (Score:2)
I've always wanted to visualize the connections between lists of open TCP/IP ports, processes, local files, and external file requests running on my Linux system. Maybe as a 3D bubble graph with each process being a sphere, and UNIX pipes being visualized as 'pipes'. The connection to the outside world would be one giant sphere containing everything, with all the TCP/IP ports being on the surface of this sphere. External file requests would appear as text labels on the outside of the corresponding TCP/IP po
Re: (Score:2)
How about a video [debian.net]?
Jeff Goldblum says no (Score:5, Funny)
Come on we've all seen independence day and Jeff looking at the 1s and 0s and then just spotting the pattern and the problem.
Jumped up security people with there fancy visualisation tools. 1s and 0s is where it is at, all you really need is a very very large green screen monitor and the force.
Personally I don't even use the monitor but instead lay hands on the ethernet cables and just squeeze out the bad packets.
Re: (Score:1, Funny)
Interestingly, I detected a worm once based on one LED blinking differently from what I expected.
Re: (Score:1)
Is that what you call it?
who's this written for... (Score:2)
he methodology is complemented with an introduction to the standard Unix-based text processing tools (grep, awk, Perl, etc). This methodology is later on applied, with a strong hands-on and how-to spirit, to an extensive set of common security use-cases, such as the perimeter threat, compliance, and the insider threat.
Do you really want someoen who doesn't know grep to be security admin? And is Perl correctly included in that list?
Re: (Score:2)
PERL? Absolutely, now go away before I replace you with a small PERL script you insensitive clout!
Re: (Score:1)
Not everyone knows UNIX, or Linux.
The live CD it comes with is based off SLAX, but should with a tiny bit of handholding be useful to Windows admins that have never used UNIX/Linux before.
I work in IT Security, I meet folks all the time, successful people, that do not know UNIX/Linux. The only UNIX/Linux in my environment are security machines that I have built for IDS and running other tools.
I have a Masters graduate, and Master candidate both from good schools that work for me, and neither one of them kne
modd parent up (Score:2)
I don't work in IT, but I do work with very sensitive data that is high risk, so IT security is an important topic and I try to understand it as much as possible.
the reason I'm shocked is that I'd expect people to at least recognize that as a blindspot in their training before they ever graduate school. Our system is exclusively proprietary so I understand not everyone would need it, but it seems like it would make sense to know it since a good seventy percen
Re: (Score:2)
No, despite it being cross-platform, Perl has very visible roots in Unix. The fact that it picked its variable notation ($variable) from Bourne Shell, or that until 5.6 the only decent way to do multi-processing was to fork(), or the fact that one of the most common idioms is the while (<>) loop that iterates over STDIN, those are all Unixisms. And those are just the ones I can name off the top of my head. If you read the Camel Book, you'll see even more Unixisms, and the History of Perl section clear
Hire Neo (Score:2)
What you do is tail all your logs to a console, sit Neo in front of said console, and have him detect changes in the patterns.
I once worked for a network security startup that had almost exactly this strategy.
They didn't do very well, I'm afraid.
Intrusion visualization (Score:3, Funny)
Correlation (Score:2, Funny)
Re: (Score:1)
Re: (Score:1)
I'm sorry, everyone.
Re: (Score:2)
It may not have been your fault - correlation is not causation.
The Spinning Cube of Potential Doom (Score:2)
Looks good (Score:2)
I have ordered one. I have doen quite a bit of SecViz myself, but a survey and reference would be most welcome.
Q1 Labs (Score:1, Interesting)
I used to work at a firm called Q1 Labs [q1labs.com]. Their founders quite brilliantly mapped aggregate network statistical information into visual presentations. In effect, the computer did what it does best: aggregate information. However their product offloaded the detection of anomalies in that aggregate information onto the human, and in particular it presented data in a way that the human brain's visual centre could readily observe patterns and deviations from those patterns (but which patterns and deviations are
Chernoff Faces (Score:2)
Did it earlier and better. See the article "The Use of Faces to Represent Points in K-Dimensional Space Graphically" or just skim the Wikipedia article [wikipedia.org]
NetGrok (Score:1)
Re: (Score:1)
Sensitive metrics first, visualization second (Score:2)
For an excellent, intellectually rigorous treatment see Andrew Jaquith, "Security Metrics", ISBN 0321349989
baobab? (Score:2)
Ever seen the program called baobab?
It makes a really nice multi-dimensional pie chart representing disk usage.
I'm sure something similar would be very useful for locating security problems such as addresses and subnets with unusual activity levels, etc.
Visualising traffic on your Internet link (Score:2)
or SPAN port. Just great: http://research.wand.net.nz/software/visualisation.php [wand.net.nz]