Researcher Resigns Over New Cisco Router Flaw 423
An anonymous reader writes "Michael Lynn, formerly a researcher for Internet Security Systems resigned today rather than conceal his research into serious new flaws in Cisco routers, according to stories at Washingtonpost.com and CRN.
Interestingly, Cisco says the the problem is not a security vulnerability, although it chided Lynn for not going through proper vulnerability disclosure channels. Both stories note that Lynn is in danger of being sued by Cisco for revealing the information, details of which were pulled at the last minute from the materials handed out to Black Hat attendees." Update: 07/28 12:23 GMT by Z : SimilarityEngine writes "Cisco and ISS are filing a law suit against Michael Lynn and the management of the Black Hat Conference, following Lynn's presentation discussing a vulnerability in IOS."
Re:I wonder... (Score:5, Insightful)
Cisco seems to suffer from the same stupidity that most other large corporations do. They'll take a report, and sit on it for weeks, and sometimes months. Full Disclosure is usually the only way to get them to actually fix the issues in a timely manner.
Cisco themselves said it was not a new flaw (Score:5, Insightful)
Later, Cisco said it was all bent out of shape because they follow an "industry established disclosure process" and because Mr. Lynn "illegally" obtained the information...
Hey, Cisco, I have news for you. "Industry established disclosure process" != "Law"
Get over yourselves, admit that you're a bunch of fuckups that can't make secure networking equipment, and move along..
Re:I wonder... (Score:5, Insightful)
The way they prefer it to go is that someone contacts them secretly, tells them the hole, and they can have it fixed all up by the time the vulnerability is published.
Then they get to look super-secure, since they were "too quick" for the bad hackers.
Some people, however, think that the only thing that'll get companies to take security more seriously is if they are actually made to look really bad, and maybe some of their products actually get hacked.
Unfortunately, when you're dealing with some giant businesses cost/benefit analysis, the only thing that can get them to take notice is a little carnage.
Is it worth it? I dunno, but it's certainly arguable.
Re:I wonder... (Score:5, Insightful)
Yes, he could. But then again, I suspect he already did. The traditional approach was to tell the vendor, and announce the flaw publicly 28 days later. That gave a vendor sufficient time to code and test a patch. However, many vendors (and Cisco seem to be particularly bad about this) sit on problems like this for several months and take no immediate action. I'd be far from surprised to hear Cisco were notified of this 3 months ago, hence Lynn's frustration and his decision to publicly talk about the flaw. I don't actually know what happened, and the above is just speculation. I suspect there's more than a grain of truth to it, though.
Re:Cisco has gone downhill recently (Score:5, Insightful)
As far as Cisco going down hill I don't really agree with that. Currently Cisco is expanding their product offerings into new unexplored territories such as IP Telephony. I have installed and supported several of these systems. As long as you follow thier design, install, and support guidelines they are as robust and as problem free as any other platform that i've worked with.
I think most people on Slashdot understand the complexities of the internet world. A minor change here can have a huge, uexpected, impact across the network or application. However, if time tested procedures for upgrades and testing are followed nothing has really changed. I think what may be giving a Cisco a bad name is all of the under qualified people out there installing their systems. The MS world of patch it, reboot, and go about your business does not fly when you critical systems are involved.
Re:I wonder... (Score:4, Insightful)
Yes, he could. But then again, I suspect he already did.
From the article:
"The decision was made on Monday to pull the presentation because we wanted to make sure the research was fully baked."
In other words, the research was not even finished yet. Isn't that a little impatient, and might there be a little chance that the researcher in question would have liked the attention he would've gotten if he presented this information at Black Hat, which was part of why he made the decision to pull out the information anyway ?
Re:This could have been avoided by using apt-get (Score:5, Insightful)
The point of buying a router is efficiency. Otherwise get a switch and a 386 running BSD or Linux... Having hardware move packets is almost certainly going to be faster (and efficient) then having a general purpose processor do it.
That said you have firmware that controls the hardware which could be "apt-get" though in reality I'd rather see an open source firmware that was also provided as binary images you could just upload.
Do you really want some MCSE throw-back building a firmware image when they can hardly manage cmd.exe?
hehehee sick.
Tom
Re:I wonder... (Score:3, Insightful)
this means it is very big, probably one of those one person can disable the whole net easily or snoop on all internet traffic without traceability.
I know of people that quit their jobs to blow the whistle and these men and women need to be held up as the heros of our time as they are the ones who not only have lots more guts that the rest of us, but are certianly more driven to not violate their core values.
I commend this man, he should be look up to.
Re:Hmmm, perhaps he needs whistleblower protection (Score:2, Insightful)
I agree that disclosure, in general, is clearly in the public interest, but this cannot always be the case.
We simply do not have enough details here to declare this disclosure "good" or "bad." Although Cisco is claiming the information was on vulnerabilities that have been fixed, that could be a PR move to stave off a stock plummet or put a stop to proliferation of the information to those that may want to use the vulnerability to bad ends.
We also can't be sure of what "fixed" truly means. How tested are these fixes? Are they complete fixes or do some variations on the vulnerabilities revealed still exist? The questions go on and on.
I'm all for protecting Whistleblowers, but only if they have done all they could to ensure that they are not causing more damage by revealing information that can still be used against current users. I'm not saying that this is clearly not the case here, only that we need more time before we declare this guy our champion.
Responsible Behavior? (Score:5, Insightful)
Cisco is actually very upfront and cooperative when you report things which might be a vulnerability (I have personally dealt with PSIRT). The people who work there are actually so polite, it's kind of annoying (I have been thanked about 2 dozen times for reporting a very minor finding).
They do however expect you to play by the rules. Even if you are the person who found a bug, you are expected to let Engineers fix the bug before you release the information.
Also, there is policy in place, which makes sure major ISPs (Carriers) are informed first, so they can do upgrades before the PSIRT release is made public.
All that makes sense, since we are really talking about essential infrastructure.
Of course, all that kind of takes away the coolness of reporting a vulnerability and you will get a lot less publicity (cisco credits you) than what you would get, if you just post to some mailing list.
If he really released information he researched at ISS without consent, well, he should face consequences. Because I obviously was to gain from it (getting a new job, making a name or himself). Hopefully he wasn't just doing it for the publicity.
Read between the lines (Score:5, Insightful)
Cisco agrees with ISS taht they're going to do something about it, but it's going to take a bunch of resesarch and time. They'll keep it quiet for a few years while they put th fix in the pipline for new models. They'll work on a firmware fix, but its back burner as long as the explot isn't public. If ISS keeps its mouth shut, they can still do work for Cisco.
Lynn hears that his research is to be hush-hush, and that Cisco will work on it, but it could be a while before there's an actual patch. No arguing that the flaw is critical will make ISS management, with a financial gun to its head, budge.
Lynn flips ISS the bird, 'cause he thinks its a major security issue, and presents his research anyway. Cisco and ISS claim they're working ont it, and that its and old flaw, and nothing really serious. And they're quietly looking for a man to fir Lynn with concrete shoes for blowing their cover.
Seems pretty clear to me.
Can't wait till car makers catch on to this (Score:1, Insightful)
Already some industries are copying the ridiculous EULA's the computer industry has come up with.
How long before other companies with something to hide start screaming about trade secrets, etc. to shut someone up?
Existing security vulnerabilities? (Score:5, Insightful)
Quote: "It is important to note that the information Mr. Lynn presented was not a disclosure of a new vulnerability or a flaw with Cisco IOS software. Mr. Lynn's research explores possible ways to expand exploitations of existing security vulnerabilities impacting routers."
Quote: "... Mr. Lynn a platform to publicly disseminate the information he illegally obtained."
If his research regards known and exsisting vulnerabilities how could they be illegal obtained? This can only happen if Cisco sits on the vulnerabilities for some time. If this is the case its a poor excuse by Cisco to state that its not a new vulnerability.
In my humble opinion its new when first made public.
If I use their routers I would like to know if they can be hacked. If they can get hacked I would like the oppotunity to take them offline if I need to protect my business.
If I don't have that oppotunity - and I loose data/values/etc due to an attack, I'll have to keep Cisco responsible.
Mod Parent Down! (Score:1, Insightful)
Full Disclosure (Score:4, Insightful)
Full disclosure is a nice cushion for people who really didnt do their job in the first place. It doesnt in no way help the users. Before the exploit is released publicly you can bet your backside its used for company spying and other shoddy activities.
A company shouldnt be afraid of scriptkiddies, theyre harmless compared to their competitors armed with their most secret info. Full disclosure makes it possible for a company to atlest try to mitigate that threat. Other disclosure puts them in the whims of the vendors.
Re:A 5 letter word for ... (Score:0, Insightful)
Re:I wonder... (Score:4, Insightful)
Personally, I'd probably rather the bank/hospital had a few weeks to establish a plan, rather than have to bang something out in an emergency, and whilst the records have already been made much more vulnerable.
Re:I wonder... (Score:3, Insightful)
Actually, yes I would. I'd much rather they fix or at least stopgap the issue instead of it sitting there wide open for all to see and/or exploit for months.
Re:I wonder... (Score:4, Insightful)
I don't buy cisco gear anymore.
Re:I wonder... (Score:4, Insightful)
c'mon... you're telling me that out of 5+ billion people on this planet, that only the person that found the exploit is the one that knows about it?
surely you're not that niaeve?
Re:I wonder... (Score:3, Insightful)
As long as it's a secret that only a few seriously malicious hackers know, the cost to Cisco is virtually nill. "Oh, your network got hacked? Well, it sure wasn't through your Cisco routers: check it out - we've got zero unpatched known vulnerabilities!" When security holes remain a secret, there is DEFINITELY a cost, but it's shouldered by the users of the product, not the designers. In general, the best way to get the designers to care is to demonstrate to the general public that Cisco is putting their networks at risk.
Not hypothetically, not a month ago, but now. Your networks are being hacked right this minute because Cisco hires sloppy firmware programmers.
Sad, but true.
Re:I wonder... (Score:5, Insightful)
Your preference suffers from the flawed (although typically wide-spread) assumtion that only one person is smart enough to discover the flaw.
If a white hat can discover it, then a black hat can too - and black hats are constantly looking. Vulnerabilities need to be *FIXED*, not discussed for weeks in private meetings.
Re:I wonder... (Score:1, Insightful)
And your preference suffers from the flawed (although typically wide-spread) assumption that having thousands of people with knowledge to exploit a flaw is no different than having one person with that knowledge.
Cisco's interesting approach to security (Score:2, Insightful)
From the (update) article:
"It is our belief that the information that Lynn presented at Black Hat this morning is information that was illegally obtained and violated our intellectual-property rights," Noh added. Lynn decompiled Cisco's software for his research and by doing so violated the company's rights, Noh said.
So, he reverse engineered their software (presumably using demonstrable decompilation techniques) to obtain all or part of the source code which he then studies to ascertain any potential vulnerabilities. Oh dear, this is a violation of their intellectual property.
Please enlighten us Cisco:
Much obliged, do take your time...
Re:I wonder... (Score:3, Insightful)
Would you consider 5 people with this knowledge "wide open"? 5000?
Re:I wonder... (Score:2, Insightful)
Using the legal system or using any type of mask to prevent or limit disclosure only helps the bottom line of the vendor.
Against security through obscurity (Score:5, Insightful)
The flaw had been privately disclosed a few months ago. Cisco, for its own reasons, didn't intend to distribute a fix before long (next year!). Too major a flaw? Publicity? Too much work already? Internal politics?
Obviously, Michael Lynn couldn't live with the idea of leaving this flaw open, and decided to disclose it publicly, thus forcing Cisco to aknowledge it and fix it. Also obviously, this wasn't the only reason. He seemed disgusted by the industry's approach to this kind of problem.
I'm always amazed by this... (Score:5, Insightful)
Real World Gray Hats (Score:3, Insightful)
Can't say for sure. But two points:
A clear case of corruption would be if Cisco tried to "kill the messenger", bury the problem,conceal its existence, so they wouldn't have to spend more resources dealing with it.
I'm not inclined to believe Cisco would do that. Rather, they'd attack the problem with as many resources as they think it deserves.
But in the real world of shades of gray it's hard to determine whether Cisco is working on the bug with all necessary and sufficient expeditious diligence, or they are needlessly and carelessly dragging their feet because fixing the problem looks to be an expensive proposition.
Personally, I think the annual reports of companies like Cisco, MS, Oracle, IBM, Sun, etc. should be required to provide an after-the-fact one-year history of their bug handling, notification, fix, distribution (with all the legal baggage that financial reporting and auditing requires), and how many of their customers' systems were vulnerable, and actually exploited (anonymous is OK there). That kind of full disclosure would provide potential customers with at least the historical information they need to make an informed decision in a functioning free market.
Re:Mod Parent Down! (Score:1, Insightful)
Moron...
Re:Mod Parent Down! (Score:4, Insightful)
*Personal* attacks should never be used, even against someone who might deserve it; it misrepresents our ideology.
However, a personal complaint about corporate policy is perfectly reasonable.
"Why is it that you, representing Cisco said that
Classic response from Cisco... (Score:2, Insightful)
It's really funny to see that quote because they ALWAYS tell you to upgrade the IOS no matter what problem is reported to them... classic response from Cisco!
Re:This could have been avoided by using apt-get (Score:0, Insightful)
Sure enough, the little company that I'd joined soon grew and grew and grew. Soon, it was one of the largest telecoms suppliers in the world. So why didn't we just increase network capacity as we grew? Well...I was so confident with my little LAN that I formally requested that the networking budget be frozen until 2012. Imagine how silly I felt when I was trying to support a worldwide organisation of over 30,000 employees with an 8-port hub! There was only one thing I could do. Yes...splice the shit out of the ethernet cables coming out of the hub, and write some advanced packet management software to handle all the multiplexed data.
My mother had an old 486SX system in her basement that she'd stopped using several years ago since it was completely fucking obsolete, but for Nortel, it was the perfect hardware solution. The only thing missing was software. I thought about the problem at hand. What do I have? TCP/IP packets flying everywhere on CAT5 cables, spliced around 7,500 times over. What do I need to do? Manage those packets. What's another name for a packet? A PACKAGE! And what manages packages? apt-get, fuck you Tom...apt-get!!!!!
I spent the next few days furiously extending the source code to apt-get to deal with TCP/IP 'packages', as well as its native currency, the .deb package. It was no trivial feat. It required some very clever hardware tricks to get it to run at full speed on the ancient 486SX hardware, including full MMX, SSE and 3DNow! acceleration. I found out later that the Intel 486SX chip didn't actually support any of those instruction sets, so I had to spend an extra day writing an emulation layer.
No matter. By the end of the week, I had my new Debian/apt-get package management system in place, busily apt-get installing TCP/IP packages across our entire network. Of course, given the restrictions of our hardware, we were bound to come across minor slowdowns from time to time. And that's what you experienced Tom. And for that I'm sorry. I really am. I could have done better. My co-workers suggested I could have used Gentoo's 'emerge' system to better optimise those TCP/IP packets better to the 486SX system. Maybe I could have. But Tom...you have to understand...I only did it because I had to. You do understand that Tom...Tom? Are you still there Tom?
Re:Why? (Score:2, Insightful)
With any large company the bureaucracy tends to slow down progress on everything. This isn't to say that fixing the problem wasn't a priority at Cisco. I honestly couldn't tell you. I also don't know exactly what is involved in fixing the problem and testing the fix so that other problems don't pop up. Microsoft has, in several cases, released a fix quickly only to find out that it causes more problems than it repaired. Then they are stuck in the embarrassing situation of having to fix the fix. People have a fit when this occurs but if someone were to take a little time they want to complain about that too.
Did Lynn do his job? Did Lynn protect Cisco and Cisco's customers? I would vote that he did not. I think he betrayed both. That may not have been his intent but I feel that is the case just the same. I think he got frustrated at how things were being handled and elevated the situation by making a Cisco vulnerability public knowledge but did that help out. One glaring problem I see is that his job did not include making strategy decisions for the company. That was someone else's job. I don't see anything in his qualifications to determine that he is the best person to determine how best to handle situations of this type. Lynn may or may not have had valid concerns but I can not imagine an instance where his actions could be justified.
Cisco may have been taking a long time to fix the problem but they had the time to spend. Now they don't. Now we can only hope that they are close to having a properly tested repair almost ready to deploy.
Handing over they keys to those that wish to hurt your companies customers is NEVER looking out for their best interests.
Re:Contact for Cisco's Point man on this (Score:2, Insightful)
Re:I wonder... (Score:1, Insightful)
I'm seriously getting sick of idiots like you on Slashdot. First you get your original premise obliterated (the assumption that two different people will discover an exploited VERY close together and that the single other black hat discovering it will use it on just as many institutions as would numerous black hats if it were publically released), now you your second premise (that fixes are instant) has been completely oblitered.
Please, dear god, just admit that you're wrong already, so people don't keep wasting mod points on you only because they don't know any better.
So now they're suing him... (Score:3, Insightful)
It must be a *really* bad hole - they might just as well hang a "crack me" sign on their heads. Either that, or they've hired security experts from Microsoft.
Re:Why? (Score:3, Insightful)
When a company is acting against the public interest in a significant way, it's appropriate to blow the whistle. Placing the entire Internet at risk of a router worm is acting against the public interest.
Of course, we don't have enough information to know if Cisco was placing the entire internet at risk, or whether they were protecting the Internet by being secretive, and it was Mr. Lynn who increased the risk. So we really don't have enough information to even debate whether what Mr. Lynn did was appropriate or not.
Maybe someone who was at Black Hat can comment?
Re:I wonder... (Score:3, Insightful)
Re:I wonder... (Score:3, Insightful)
We've seen this over and over again historically - if there is no disclosure, there is no urgency, so the problem remains unpatched until the worm hits, and then suddenly, after the fox is done raiding the henhouse, steps are taken to close the door.
I don't know if that is the case here - I really have no information at all about the vulnerability, and TFA doesn't tell us anything substantive. But that's the argument for rapid disclosure. The usual rule is to give the responsible party notice, and wait a while to see if they fix it. If they don't, disclose.
If that's what happened here, I'd say Mr. Lynn did the right thing. But again, we really don't know, at least based on TFA, whether that's what actually happened.
Re:Why? (Score:3, Insightful)
-Hope
Re:Contact for Cisco's Point man on this (Score:2, Insightful)
There is also specific state laws concerning NDAs and trade secrets, see:
http://www.michbar.org/e-journal/bar_journal/bppj
http://www.nolo.com/article.cfm/ObjectID/2ECF62E6
But hey, if want to believe that violating things that exist in the law books isn't illegal, go ahead.
In any case, I'd question the validity of an NDA which required somebody to keep secret a piece of information contrary to a large public good.
It's a good thing that you're not a judge nor lawyer then, because you can't violate an NDA just because you think it's not doing the public good. "Hey, I believe that keeping this technique for making super cheap LCD screens is against the public good, I'll just reveal it!"
For example, if I found out under an NDA that my employer was putting out a product that was killing people, and keeping it quiet, I'd be ethically bound to blow the whistle.
So Cisco is killing people? What's your point?
Certainly an NDA that forces you to break the law (such as by concealing knowledge of a crime) would be void.
What law is the NDA in question forcing the person to violate?
However, I would feel justified in doing so if I had clear evidence that an employer was committing a crime, or harming people and not doing something about it.
So do you actually have any reason to believe that Cisco/ISS are comitting a crime, or is that just 100% wild, rampant speculation?
What idiots modded this thread informative? (Score:5, Insightful)
Four months ago.
However, the more damningly flawed portion of your argument is that 'now Cisco doesn't have time to fix the problem'. <snort>
Could you please provide proof that this flaw hasn't been actively exploited since even before the time at which Lynn found it?
It is, needless to say, impossible to prove a negative.
But (Score:3, Insightful)
For all he know, it's been exploited for weeks.
Ideally, we could say here is an exploit. In a week I'll release it to the public. Unfortuanatly, he would get sued, and the exploit would go unpatched for a while.
Re:Why? (Score:2, Insightful)
I thought your post was well reasoned and interesting, but I had a problem with this part. You might want to consider that as a member of a society, particularly a democratic one, where in theory we're all (US citizens for US, but if you believe in a democratic world governance then as a citizen of the world as well) the top level of government. As such you have a responsibility or loyalty to the society you belong (family, friends, neighbors, etc...) before a loyalty to an employer. Exaggerated out, your statement makes the appropriate response when a company has you physically damage people (poison the water or even out right murder) be loyalty to the company first.
Historically, worms follow patches (Score:3, Insightful)
Still, most exploits seem to be reverse-engineered from patches. Compare the patch to what came before and you have a serious clue to the problem.
That's in the public world; I don't claim to have any insight into privately held 0-day exploits. I suppose that a there are some blackhats as clever as the white, with equivalent labs.
Re:I wonder... (Score:3, Insightful)
What's dumb is that people sign NDA's and then reveal what they learn. Even if Lynn didn't have an NDA personally, ISS almost certainly did, and he would have been bound by it. In addition, some of the information may have been based on ISS trade secrets, and since he's no longer an employee, he would have no authority to discuss them. So, in this case, a civil lawsuit is absolutely appropriate.
If you and I have a contract that you won't disclose X without my permission, and you tell me you are going to disclose X, what should my reaction be?
Trains, planes, and software (Score:3, Insightful)
All corporations (I'm talking about large corporations with hundreds or thousands of employees) are like trains, planes, or other large pieces of equipment. They can not stop and/or turn on a dime. (As the saying goes.)
As in my previous posting on this subject - think of a bus which is going madly down the road at 100mph. Within a mile of where the bus (ie: the company) is is a bridge which has collapsed (ie: the problem). If you start a mile back from the bridge you can easily stop the bus and save everyone (ie: anyone who uses the company's product). If you wait until there is only 1/2 of a mile the bus can still be saved but they might have to slow down a lot faster and they could blow some tires and maybe have an accident. (Thus hurting some of their customers.) Or you could wait until there is only 1/4 of a mile and try to stop the bus. Here, since a bus travelling 100mph travels 100 * (5280ft/60/60) = 146.6666ft per second. It means that the bus has less than 10 seconds to stop. Most porbably, unless the bus driver causes the bus to fall over onto its side - the bus will most likely go over the bridge and kill everyone.
The same holds true for talking about problems in ANY WAY, SHAPE, or FORM when it comes to computer software or computer hardware. You can't just jump out there and start screaming there is a problem because the bus can't stop that fast to prevent disaster. Nor can you tell a company about a problem, wait a couple of hours, days, or even weeks and get mad because nothing has been done. It takes a while to bring the bus to a stop, pick up on what you have to say, and then to start back up again.
What's a good rule of thumb? Three to six months minimum depending upon how severe the problem is. If it is just a one or two line coding problem - three months. If it is a major change due to parts of a program having to be either completely re-written or major portions having to be changed - six months. And remember - that is a MINIMUM requirement. Normal length of time to fix? More probably two to three times those minimums. That's because you are not the only person who may have found a problem as well as the fact that they are trying to put in new features that have been requested. The same people work on both things at the same time.
So people who find problems need to think in months - not weeks, days, hours, minutes, or seconds. Because that is how long it will take to fix a problem. In fact, sometimes something that looks really simple turns out to be a real mess to fix. It all depends upon the way in which some software was originally written. So you can't base how fast the company fixes something by what you may think is a fair amount of time. You just need to be patient while the company does what it can to fix the problem.
Now, as for the company - it is extremely important for companies to keep everyone up-to-date on any/all progress made to fix a certain problem. This can even be automated somewhat. But it is very important not to try to hide the problem because as anyone knows - that is what gets a company in trouble. Trying to hide things that is.
Comment removed (Score:2, Insightful)