Microsoft Fails Antivirus Certification Test (Again), Challenges the Results 228
redletterdave writes "For the second time in a row, Microsoft's Security Essentials failed to earn certification from AV-Test, the independent German testing lab best known for evaluating the effectiveness of antivirus software. Out of 25 different security programs tested by AV-Test, including software from McAfee, Norman, Kaspersky, and others, Microsoft's Security Essentials was just one out of three that failed to gain certification. These results are noteworthy because Microsoft Security Essentials is currently (as of December) the most popular security suite in North America and the world."
Re:This is why (Score:5, Interesting)
I'm anything but a Microsoft lover, but I have to defend them.
About a million years ago, back during the DOS era, a friend and I wrote an anti-virus suite (the ARF Antivirus, maybe you can still find it online, though I don't recommend that you use it!). It was quite effective; we used the file integrity approach, and stored the integrity information in the files themselves. (We were up front about it; some people don't like that, so we said, hey, you don't like it, just don't use our stuff. No hard feelings.)
Ergo, I think I can at least offer an opinion that's slightly above drooling moron status.
One of my biggest complaints about AV tests is that they're unrealistic. This has been years ago, now, so maybe it has changed, but back then, the folks who did the testing were arrogant and very hard to deal with. Your software had to produce a .TXT log file; it had to do this, it had to do that, or they would just fail it outright.
Once you made them happy, then they tested it against every virus they could find, including some that WERE NOT (and never would be) in the wild.
Bottom line, and to make a long story short: the people who were writing AV software back then were writing it for these tests, and not for the real world. I don't know if that's the case nowadays; I just don't know. (For that matter, maybe Microsoft's stuff really does suck. Given how badly their stuff worked back in the DOS era, it wouldn't surprise me. But I just don't know.)
But fair is fair. I ran from that circus after about a year of endless arguments with the pompous egotists in Compuserve's Anti Virus forum. I don't know if it's still that way, but I haven't used anyone else's anti virus stuff in years (I protect my stuff a different way, primarily by using secured Linux with good backups, and with periodic integrity checks).
Correct you if you're wrong, but... (Score:5, Interesting)
So long as you keep your software updated then there's not really much of a point other than the chance you'll spread an infected file onward without being infected yourself.
Think. No, that's not good enough, think some more: Viruses (we are explicitly talking viruses here, says "Antivirus" right in the test and headline) exploit unpatched vulnerabilities (mistakes) in software. Patched software is immune to the prior vulnerabilities, so AV won't "protect" you from things you're immune to. It also won't protect you from viruses with signatures that it doesn't know about. So, What's the point of wasting all those CPU cycles scanning? Oh, maybe you got infected and it could remove it later? WRONG. Viruses actually mutate, say a malware author snags a virus, they reverse engineer how the payload is delivered and they change the payload to theirs and send it on its way -- The malware can even install other malware once it gets running. So, the (automated) removal options/instructions are probably not complete if the code has ever had a chance to run before. Ah, so now you may be thinking that it's exactly the reason why you'd waste CPU time on an AV scan, to detect infection so at least you'll know -- Except that's just silly. Think. If you were a spy and I asked you if you were a spy then would you say yes? An AV running in an infected machine can not reliably determine the state of the infected machine. AV: "Any Viruses here" Virus: "Nope!"
Often times I'll get people telling me, no matter which AV product they're using, that their machine is working strange, slower, showing adverts and wrong websites, and their AV will be chugging along saying everything is fine. You get more reliable warning from the malware itself! "You may have been Infected with 2042 viruses!" the scareware will prompt every boot, while Norton, or McAfee, or AVG, or ANY AV product I run across the infected machine says the coast is clear. You can't "remove" malware -- Nuke it from orbit, and re-install, it's the only way to be sure.
Look, people, hardware supports virtualization now. If you're NOT running your Windows boxen in a VM, then you're not concerned enough about security to benefit from an anti-virus anyway. Boot from a known clean state, maybe even a LiveCD/USB then do your virus scanning from there if you want to be able to detect anything with any degree of certainty, and even then it's questionable. If your data partition is separate from your (virtual) OS partitions then you can just always run (or restore) from a known good snapshot, and install updates to the known good snapshots, then make another snapshot before you do anything else.
I'm no Microsoft apologist, I don't have to worry about such things as much anymore because I use an OS that gets the patches out much faster than MS does, but I can certainly see where the people who understand the issues in Microsoft might realize that Antivirus isn't really the right option anyway, it's just a waste of time and there are other better solutions... Windows Steady State (or whatever it's called now), for example.
"Insanity: doing the same thing over and over again and expecting different results."
"The significant problems we face can not be solved at the same level of thinking we were at when we created them."
- Albert Einstein
Re:That site is BS (Score:4, Interesting)
Reading the 2012/2013 results for Protection only:
BitDefender
F-Secure
Trend Micro
Get 6 out of 6.
Shady AV companies (Score:4, Interesting)
Re:My response in 3 words (Score:4, Interesting)
> Microsoft DOS 6 with AV built in ... was defeated by every virus writer
That's because MSAV included the classic, textbook example of "security through obscurity." Utilities like FORMAT and FDISK would do the same things as some malware, which would cause false alarms. The users would be terrified by this, so there was a solution: a "secret" (wink, wink!) system call in the OS that their utilities used to temporarily disable the alarms. (!!!)
It was top secret ... so naturally, everyone knew about it. A call to disable VSAFE became one thing that EVERY DOS virus writer put at the top of his code. Naturally. Of course.
Ah, you're bringing back memories now. :)
Re:Shady AV companies (Score:5, Interesting)
Kapserasky was accused of this when it was noticed that their definition files contained signatures for some zero-days that hadn't been seen in the wild yet.
Re:This is why (Score:5, Interesting)
I'm not surprised at all.
Our approach was to stop viruses before they got onto the computer. I remember Wolfgang(?) with Integrity Master (another system available at the time) complaining of the same thing we did: the "AV shootouts" focused entirely on scanners.
They were easy to test! Just turn them loose on a hard drive full of virus samples and see how well they did! But what about people like us that took a different approach?
Our ARF system not only "innoculated" the executable files, I can give away some of our secrets now. (Heh. Like it matters.) I actually became a DOS "guru" and figured out ways to hook into the OS itself. We watched the SHARE hooks, too -- an obvious vulnerability that everyone else ignored. We hooked all of the standard interrupts *inside the kernel* (we didn't just patch into the interrupt chain), we captured the "trace" interrupt to see if anyone was "tunneling," we did CRC "checksums" on the actual DOS code and other key areas.
I'm not boasting, but we never, ever found a virus that could get past us. The worst case, the system would get confused and hang, but there would be no infection. After reboot, the system was still clean.
Now ... how do you test that? How do you "shoot that out?" You don't. These so-called testers love scanners. SCANNERS! That's all they want to test.
That, combined with the fact that virtually no one registered it (and the additional fact that Windows 95 had come out), made us lose interest. I briefly worked on moving the blocker into a VxD, but it wasn't worth the bother.
Re:This is why (Score:5, Interesting)
But I'll also add this condemnation of Microsoft. I haven't traced through their OS in many, many years, so to be fair to them, things like this may no longer be the case. But back in the day, they were *notorious* for repackaging the same code over and over and over. DOS was well-understood by that point and its vulnerabilities were well-known and easily exploited.
All because Microsoft couldn't even be bothered to reassemble or recompile key parts of the kernel.
For example, I did one of the first analysis (analysees?) of the so-called "antiexe" virus. DOS 5 through DOS 6.22 were so similar, the freakin' offsets in the kernel didn't even change(!). The entry point to the DOS kernel was in the same exact location in all. Antiexe simply looked up the DOS data segment address, then started poking in junk at the *fixed* (and known) offset of the entry point of the kernel. That way, it could bypass most current security software. (But not ours. Grin.)
Our system also addressed a killer bug (first discovered by Geoff Chappel) that Microsoft had known about, but had apparently not bothered to patch: if the partition table was recursive -- i.e., an extended table pointed back to itself -- the computer would hang during the boot. Even booting onto a floppy wouldn't work! As soon as the kernel on that floppy started trying to examine and mount the hard drive's partitions, it would loop forever. Hang tight.
I can't even imagine how many people carried their computers into a shop, only to have the tech tell them that their hard drive was defective. (I know of a couple of cases myself.)
So ... believe me when I say I'm anything but a Microsoft lover. Like I said, maybe they've improved now, but back in the day, they were making money hand over fist and couldn't even be bothered to address obvious stuff like this.