Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Spam Books Media Book Reviews

Ending Spam 184

Posted by timothy from the overdue dept.
Shalendra Chhabra writes "Jonathan Zdziarski has been fighting spam since before the first MIT spam conference in 2003, and has now released a full-on technical book, Ending Spam, on spam filtering. Ending Spam covers how the current and near-future crop of heuristic and statistical filters actually work under the hood, and how you can most effectively use such filters to protect your inbox." Read on for the rest of Chhabra's review.
Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification
author Jonathan A. Zdziarski
pages 312
publisher No Starch Press
rating 8
reviewer Shalendra Chhabra
ISBN 1593270526
summary Very Good Book Covering Statistical Models and Techniques Implemented in Current Spam Filters


Spam (unsolicited commercial email) and phishing (fraudulent emails) are causing losses of billions of dollars to businesses. Many initiatives are currently underway for fighting this challenge. On the legal front, a Virginia court recently sentenced a prolific spammer, Jeremy Jaynes, to nine years in prison, and a Nigerian court sentenced a woman to two and a half years for phishing. Michigan and Utah have both passed laws creating "do-not-contact" registries in July/August 2005, covering e-mail addresses, instant messaging addresses and telephone numbers. Technical initiatives to fight spam include server- or client-side spam filtering, using Lists (Blacklists, Whitelists, Greylists), Email Authentication Standards (IIM, DK, DKIM, SPF, SenderID), and emerging sender reputation and accreditation services.

Ending Spam is the first book explaining the fine details of the theoretical models and machine-learning algorithms implemented in these filters. The book is divided into three parts: introduction to spam filtering, fundamentals of statistical filtering, and advanced concepts of statistical filtering.

The first section of the book discusses the history of spam, spam kings, different approaches for fighting spam such as blacklisting, whitelisting, heuristic filtering, challenge response, throttling, collaborative filtering, Authenticated SMTP, Sender Policy Framework and SenderID, spammer fingerprinting, etc. However, the author omitted any mention of locally-sensitive hash functions (such as Nilsimsa Hash) to counter spammers' random insertion of words, the use of CAPTCHA (Completely Automated Public Turing Test to Tell Computers and Humans Apart), Greylisting, Identified Internet Mail, and Domain Keys (now Domain Keys Identified Mail).

In the next chapter, the author clearly explains various components of a Language Classifier Pipeline, including the Historical Dataset (aka wordlist, database, dictionary, filter memory), Tokenizer, and the Analysis Engine with its feedback loop. However, the process flow of a language classifier could have been more generalized, e.g. incorporating an initial text-to-text transformer. This chapter also covers the advantages and disadvantages of various training modes for filters, such as Train Everything (TEFT), Train-on-Error (TOE), and Train Until No Errors (TUNE). This part concludes with the description of Paul Graham's famous spam-filtering technique using Bayesian classification (as described in "A Plan for Spam"), Gary Robinson's Geometric Mean Test, Fisher-Robinsons Inverse Chi Square (including the source code for the inversion function), and some other tricks for optimizing spam- filtering accuracy.

The second part of this book deals with the fundamentals of statistical filtering. The author explains HTML and Base64 encoding, followed by a detailed description of tokenization techniques (e.g. Sparse Binary Polynomial Hashing). Then there's a discussion of the various tricks that spammers use for penetrating filters. Although these tactics are mentioned in John Graham-Cumming's "Spammers Compendium," Jonathan has very elegantly explained why some tricks work for spammers and some don't. This part concludes by addressing some of the resource, storage and scaling concerns raised by the large number of features generated from tokenization techniques.

The third part of this book deals with advanced concepts of statistical filtering. This includes the testing criteria for measuring accuracy of an email filter, and some advanced tokenization concepts, e.g. chained tokens (taking word-pairs and phrases into account, instead of individual words) generated using a sliding 5-byte window as mentioned in Sparse Binary Polynomial Hashing. The next chapter describes the Markovian Model implemented in the CRM114 Discriminator, but the author fails to describe different weighting schemes for features implemented in the Markovian-based version of CRM114. The author then describes the Bayesian Noise Reduction Technique for purging "out of context" data from the mail text. This chapter concludes with a very nice summary of collaborative algorithms and techniques, such as Message Innoculation, Streamlined Blackhole List, Fingerprinting, Automatic Whitelisting, URL Blacklisting, and Honeypot email addresses for snaring spammers' address harvesting bots.

The most interesting part of this book is the appendix, where the author presents interviews with John Graham-Cumming of POPFile, Brian Burton of SpamProbe, Marty Lamb of TarProxy, Bill Yerazunis of CRM114 Discriminator, and Jonathan Zdziarski of DSPAM (himself). I loved this section.

The salient points of the book: it's very easy to read; each chapter begins with a very thought-provoking introduction, and concludes with a crisp "final thoughts" section. The number of technical errors are very few in this print, and the illustrations are of good quality. Since the book is geared more toward the Bayesian and statistical generation of spam filters, the absence of certain spam-busting technologies is acceptable. However, a noticeable omission is the lack of discussion about measuring spam-filter accuracy, and what impact this has on setting filtration thresholds. A section on the economics of tradeoffs, and the use of a Receiver Operating Characteristic curve (ROC) would have been very helpful.

Overall, by putting together Ending Spam, Jonathan Zdziarski has made another significant contribution (after DSPAM) to the anti-spam community. Whether you are a system administrator, anti-spam researcher, engineer or a newbie interested in fighting spam, this book is a great reference.

William S Yerazunis and Richard Jowsey also contributed to this review. Shalendra Chhabra is a Graduate Student in Department of Computer Science and Engineering at University of California, Riverside. He is on the development team of CRM114 Discriminator and has presented his work at MIT Spam Conference 2005, Cisco Systems, and Stanford University. You can purchase Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
This discussion has been archived. No new comments can be posted.

Ending Spam More

Ending Spam

Comments Filter:

  • You can't have both... (Score:3, Insightful)

    by TarryTops ( 888130 ) on Monday August 15, 2005 @05:29PM (#13325276) Homepage
    The openness eill have to pay it's cost. and spam is one such pest. You can develop better strategies for pest control. But in the end it's a trade off.

  • Bill Gates promised to end it (Score:2, Funny)

    by Anonymous Coward
    Why worry about spam? Bill Gates promised to end spam by early next spring. (It's marked in my calendar along with the link to where he promised, but not with me in my PDA right now.)
  • I'm wondering... will UCE (Spam) be like malaria... controllable in most areas but impossible to eradicate?

    Or will these dedicated folks and others be able to eliminate it, perhaps by changes to the mail protocols?

    • Is spam a parasitic malady and, if so, what next? (Score:4, Insightful)

      by WillAffleckUW ( 858324 ) on Monday August 15, 2005 @05:41PM (#13325390) Homepage Journal
      I'm wondering... will UCE (Spam) be like malaria... controllable in most areas but impossible to eradicate?

      Or will these dedicated folks and others be able to eliminate it, perhaps by changes to the mail protocols?

      Interesting question that, considering my work involves malaria.

      My guess is that, like malaria and most parasitic infestations, we will at some point develop a "cure". The "cure" will work for a few years, after which the parasite (spam) will have adapted, surviving until then in different hosts (old windows machines donated to Africa, who knows). Then, having developed a new trick, it will come back as strong as ever.

      Biology teaches us that organisms adapt to changing environments, thru selective breeding (natural), point mutations, and unforseen combinations (see the H51N avian influenza). We can develop cures, but once we do so, we can be fairly sure that, baring species extinction, it will develop methods to cope with our cures.

      An easy solution would be to move to IPv6 - but this, like authentication, will only kill off the spam which doesn't use "trusted email clients that are identified" while the spam that can survive will be encouraged to spread like wildfire.

      So long as the fiscal, legal, and societal penalties for spamming are fairly low and the rewards are high, and while most people do nothing about it, it will spread.
      • Bad analogy. Spam is not an organism or infection. It is a business model. It does not "survive" in computers, but in a combination of economical, technical and legal conditions. Once those conditions become strongly unfavorable to the business model, there isn't really much that adaption can do. Selling "snake-oil" wonder cures used to be a really big, widespread business model. Better-informed consumers and increased regulation of the market for medicine have all but eradicated this practice. It survives,
        • Bad analogy. Spam is not an organism or infection. It is a business model. It does not "survive" in computers, but in a combination of economical, technical and legal conditions.

          True and False.

          Spam acts like a parasitic organism, due to the favorable conditions for the business model. It does, in some cases, actually "survive" in certain computers, which are spam zombies that spew out spam from a spam source - in fact, there are a few at the other UW (in Wisconsin) which utilize the identified computers th
          • It does, in some cases, actually "survive" in certain computers, which are spam zombies that spew out spam from a spam source

            That's not survival in the "organism" analogy, since a zombie will not send spam without a source, which will be gone when the business model is not workable, and especially not cause new source to appear.

            like the malarial mosquito, spam uses those responders (infected persons) to download more spam zombie software, since they tend not to be technical enough to remove the infection.

            Yo
            • I totally agree that computer worms/viruses work very much like an infectious disease. But they are merely one tool that spammers use, not identical with the phenomenon of spam as such.

              Just as a mosquito is merely a tool the malarial parasite uses to spread itself.

              Let's say we knock out something that permits mosquitos to infect human hosts. Chances are that it might only partially impact malarial infections of non-human hosts. The impacted malarial bug, provided it survives and breeds, may then decide to
              • Just as a mosquito is merely a tool the malarial parasite uses to spread itself.

                Except that spam does not use zombies to spread itself, SPAMMERS use zombies to spread spam.

                Your analogy is simply flawed. Spam is NOT an organism. It does NOT "survive" somewhere, adapt and spread from the places where it survived.

                And we certainly DO go for "species extinction", by eliminating the conditions that make spam practicable and profitable. You enumerate some of those conditions yourself in the end.
                • Except that spam does not use zombies to spread itself, SPAMMERS use zombies to spread spam.

                  Your analogy is simply flawed. Spam is NOT an organism. It does NOT "survive" somewhere, adapt and spread from the places where it survived.

                  And we certainly DO go for "species extinction", by eliminating the conditions that make spam practicable and profitable. You enumerate some of those conditions yourself in the end.

                  If it looks like a duck, and it quacks like a duck, and it paddles like a duck, you want me to chec

        • Re:Is spam a parasitic malady and, if so, what nex (Score:4, Interesting)

          by jonbryce ( 703250 ) on Monday August 15, 2005 @06:44PM (#13325896) Homepage
          Spam may not be an organism or an infection, but the people who send it are. So I think it is a perfect analogy.
  • While all of these different technological approaches to spam are worth pursuing, they just don't build the same esprit d'corps as a mob with pitchforks and torches at midnight.

    • Re:Esprit d'Corps (Score:2, Insightful)

      by DavidTC ( 10147 )
      Don't be silly.

      Mobs attacking spammers should only be armed with plastic spoons. All fourteen million of them.

      Remember, if you only poke them once, it's not only not murder, it's not even assault, and perfectly legal under the CAN-POKE-SPAMMERS act, as long as they have a 'business relationship' with you, which they obviously created by spamming you.

      And, to make it fair, they are allowed to opt out of any member of the mob poking them. One at a time, in writing, and we'll even waive the 48 hours to proce

  • "Jonathan Zdziarski has been fighting spam since before the first MIT spam conference in 2003,"

    Awww, poor babies. That's a long time to fight spam.

    • Re:Sorry for the flamebait but (Score:5, Informative)

      by Stanistani ( 808333 ) on Monday August 15, 2005 @05:37PM (#13325354) Homepage Journal
      From:
      HERE [castlecops.com]

      "ABOUT THE AUTHOR:
      Jonathan A. Zdziarski has been fighting spam for eight years, and has spent a significant portion of the past two years working on the next generation spam filter DSPAM. His research in algorithmic theory and neural networking has led to the development of many new approaches in language classification, and he has played a key role in designing some popular algorithms in use today, including Message Inoculation, Bayesian Noise Reduction, and the first functional Neural Networking algorithm for spam filters. Zdziarski lectures widely on the topic of spam and was a speaker at the 2004 and 2005 MIT Spam Conference.
      "

  • The best way to fight spam (Score:5, Funny)

    by WillAffleckUW ( 858324 ) on Monday August 15, 2005 @05:35PM (#13325327) Homepage Journal
    is with a knife, a spatula, and a frying pan, preferably over a hot wood fire.

    Yum!

  • Score -5 Outdated. (Score:2, Insightful)

    by Anonymous Coward
    As with any book of this type, it is outdated by the time it reaches the shelves. The spam battlefield changes on a daily basis and the tools used to fight the battle, change with it daily.

    By the time a book has been written edited, proof read(though many publishers skip this part), type set, printed, distributed and sold, it no longer resembles the technology.
    • Sure, some details will change, and spammers and anti-spammers will pick up new tricks and abandon old ones, and the percentages of email that are spam will keep changing (normally up, but I saw one recent article saying it had dropped significantly in the last year.) But most of the fundamentals don't change much, or at least not very fast. Filtering techniques, Bayesian analysis, collaborative filtering, etc. are a solid core of knowledge that will continue to be useful.

      Rule 1 (Spammers always lie) wo

      • No, Rule 1 is "spammers lie". Rule two is "see rule one". Rule three is "spammers are s-t-o-o-p-i-d".
  • Spam will continue to disguise itself as legit email. You can try to filter it out and set more strict filters but catching legitimate mail is far more likely to happen. In the end, you have to make a trade off and practically accept some spam.
    • The root problem is with SMTP. We can try to patch it up with SPF and SenderID, we can try to find ways of putting identifiers on emails, but at the end of the day the protocol itself was built in a simpler age.

      The ultimate solution will come when we move to a new-generation mail delivery system. But the day is a long ways off, because the sheer cost of implementing such a system and the necessity of having it integrate with older SMTP systems for the years required for large-scale adoption means that spa

      • Re:You can't catch it all (Score:5, Interesting)

        by plover ( 150551 ) * on Monday August 15, 2005 @06:07PM (#13325614) Homepage Journal
        You've missed the last two years in spammer technology, haven't you?

        Spam is no longer simply the domain of a giant server with a huge database. It's increasingly being sent out by zombie PCs, infected with viruses or trojans. Spammers pay the zombie-farmers to send their crap. Zombies send the email masquerading as the PC owner, using their credentials. Sender-ID? No problem, he's got one. SMTP? Sure, use the victim's server.

        Zombies mean that no matter what technology is used for sending validated, signed, pre-paid, whatever email, the zombies will have access to those resources and will still spew their crap. No anti-spam server technologies are going to prevent Windows machines from getting infested.

        • Re:You can't catch it all (Score:5, Interesting)

          by MightyMartian ( 840721 ) on Monday August 15, 2005 @06:17PM (#13325684) Journal
          I'm well aware of the zombie problem (having been the recipient of very nasty distributed dictionary attacks). The way that mail ought to work is that any system without an MX record ought not to be permitted to send email to an MTA. Unfortunately for a variety of reasons (from legitimate to pure incompetence or laziness) many mail servers do not have MX or reverse records, and because sufficient amounts of legitimate email come from such servers, and because there is no line drawn between MTA and MUA (all go through port 25TCP), zombies can quite happily spread havoc.

          The first step to a new mail system is to assure that only legitimate and properly configured mail servers honoring MX records on outgoing mail (or whatever ends up replacing MX records) can expect delivery. Mail admins' hands are tied by stealth systems or badly configured ones, but if we do try to implement the no-MX rule, which would eliminate the zombie attacks, we end up shutting out systems that, for whatever reason, don't publish an MX record for outgoing servers.

          Zombies ought to be the easiest thing to shut down by a) not permitting non-MTA machines to push anything beyond the network via port 25 and b) publishing both incoming and outgoing mail servers.

          • I saw this somewhere else, and I liked it... so, here goes:

            Your post advocates a

            (*) technical ( ) legislative ( ) market-based ( ) vigilante

            approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

            ( ) Spammers can easily use it to harvest email addresses
            ( ) Mailing lists and other legitimate email uses would be
          • Trouble is that a zombie has access to the user's legitimate mail system, which they can abuse.

            In the end, no technical solution is really going to solve it; you're using "is this machine meant to send mail?" as a heuristic for "is this mail junk mail?". As you can't define junk mail objectively, in computer-friendly criteria, any filter is inevitably going to make mistakes. The only question is whether your filter tends towards false positives or false negatives.

  • Ending Spam? (Score:5, Insightful)

    by demonbug ( 309515 ) on Monday August 15, 2005 @05:39PM (#13325371) Journal
    Does anyone else find it funny that a book called "Ending Spam" talks about spam filtering? Maybe I'll go write a book; "Ending World Hunger: How To Filter Sally Struthers From Your Television".

    If you can't see it, it ain't there?

    • Re:Ending Spam? (Score:3, Insightful)

      by DogDude ( 805747 )
      Well, I think that most rational people would understand the title to mean "Ending spam as it pertains to ME". In which case, as far as most people are concerned, if they don't see spam, then the spam problem is solved. I really don't think that that is an inordinate amount of literacy license.

      And yes, if you don't see it, then unless you're a system administrator (can't be more than 0.001% of the population), the problem IS solved. The problem isn't spam per se, but that spam clogs up MY inbox.

      It'

    • Effecitve filtering will end spam (Score:5, Insightful)

      by Sycraft-fu ( 314770 ) on Monday August 15, 2005 @05:52PM (#13325491)
      The reason spammers do it is that their message reaches people, enough of them to make it worthwhile. So, the more effective and widespread the filters, the less messages that reach people, and the less it's worth. If the filters were really effective, nearly 100%, it would simply not be worth it to spam, you wouldn't make any money because no one would see your message.

      I don't think we'll ever get there, but yes filtering really could end spam.
      • Reminds me of the conversation at the end of Batman Begins with Gordon and the Bat:

        Gordon: "Batman making a stand as he has will only escalate the problem."

        If suddenly the masses are educated on spam filtering, wouldn't spammers just adobt tactics to avoid them?

        I mean it is afterall a "spammers market". They have increased resources because they're getting all the money. I'm sure the spammers are much smarter than most techies who use filters, they just don't care. They think, "If this techie is going to us
        • If suddenly the masses are educated on spam filtering, wouldn't spammers just adobt tactics to avoid them?

          But that's exactly what we've been seeing over the years.

          Granny has never filtered a spam in her life. The ISPs have taken up automated spam filtering on her behalf. That's why the spammers can't stand still and let just us techies filter their sludge. The techies took the fight to the next level, blocking spam further up the chain so the benefits of spam-blocking translated to everyone. Thus,

        • If suddenly the masses are educated on spam filtering, wouldn't spammers just adobt tactics to avoid them?

          That's why the solution has to treat the evasion of spam filtering like any other sort of computer cracking (i.e. a federal offense resulting in a few years of PMITA prison).

      • There are an infinite number of people who find an ad and give spamers thousands of dollars to send out their ad to millions of people. The rich spam bastards don't make money selling pills (even though some have admitted to it), they make their money by reselling spaming services to people who think they will work for their product.

        The only real way to stop a spamer is jail or a baseball bat but someone else will always be in line to replace them.
        • Your post advocates a

          ( ) technical ( ) legislative ( ) market-based (*) vigilante

          approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

          ( ) Spammers can easily use it to harvest email addresses
          ( ) Mailing lists and other legitimate email uses would be affected
          ( ) No one will be able to find the guy or collect the mon
          • I wasn't advocating a solution, just describing the what I've heard about.

            I know of 3 cases where spamers have stopped (at least for a bit) doing their dirty work where a baseball bat (or similar) were used. The results are one dead Russian who won't be spaming any more (but his friends have taken over his work), A Kiwi that is now back spaming even though there are rumors that his kid has taken at least one beating for his activity, and one Aussie spamer that decided that a few people showing up at his
      • The folks smart enough to figureout the spam filters would not be clicking on spam messages in the first place. So who is this book addressing? Is this providing a tech solution to a social problem?

        I agree with the original post -- this is not 'stopping spam'.
      • If we successfully filter out 99% of the spam, wouldn't the spammers just have to send out 100 times more spam to make up the difference?

        If only that were the case... Really, they'd just have to figure out what is special about the 1% that gets through and make all the spam look like that.

        Spam filtering is like DRM. Somebody wants to bypass it badly enough that they will.
      • The reason spammers do it is that their message reaches people, enough of them to make it worthwhile.

        I disagree. Spammers aren't advertising their own products, they're advertising the products of suckers who beleive that it is worthwhile to pay the spammers to send their spam into a void. Even if all the spam in the world was filtered with 100% effectiveness, there would still be greed fueled suckers who beleived that a spammer could bring them customers for their phony erection drugs and porn link site

      • The response of spammers to better filters has been to send more spam. You might not see it in your inbox, but you _will_ see it in increased usage fees.

    • Re:Ending Spam? (Score:4, Insightful)

      by pomo monster ( 873962 ) on Monday August 15, 2005 @05:56PM (#13325523)
      Well, in a way, and I don't mean philosophically. If nobody can see the spam, then it really will dry up--spammers won't even bother.

      There's no such thing as a perfect filtering system, but for every message blocked, that's extra effort for the spammer to get through, making it less and less worthwhile to spam at all.

      Or maybe they'll just send more and more, hoping at least one gets through.
  • We spammers love you idiots who use spam filters. You were never going to buy from us or fall for our scheems anyway, so you do extra work to filter your e-mail and that way we are not bothered by you reporting us or attacking us. We are free to continue to waste your bandwidth and overflow your inbox, but you never see the spam and you leave us alone, to keep spamming those too ignorant to protect themselves. The complaints die down and we get what we want, the unknowing victims. What a great system.

    Heck

  • Email is mostly broken (Score:4, Interesting)

    by mcrbids ( 148650 ) on Monday August 15, 2005 @05:42PM (#13325397) Journal
    Email, as a system, is fundamentally broken. It's this broken design that allows SPAM to happen in the first place.

    Current anti-spam solutions are to email what an Antivirus package is to Windows - a hack add-on that increases complexity and costs without solving the underlying problem(s).

    Rather than fight viruses, we should be engineering an O/S that's inherently resistent to them. How many of you Linux/BSD/MacOS users EVER use antivirus, or need to?

    Rather than build ever-better antispam filters for Email, we should be engineering an email solution that's inherenly resistant to SPAM.

    The answer lies in authentication - who is sending the email. Some of the best technologies now available use degrees of authentication without actually *saying* it outright. Examples are: refusing invalid domains, greylisting, challenge-response, SenderID - all of these are some form of authentication.

    As these are, one-by-one bypassed by the spammers, the need for authentication of senders will continue to increase, until the dolts who will invariably reply with that "your solution will not work because... (check the options)" are shown to simply be.... wrong.

    Give it time. It's already happening whatever the originators of the SMTP protocol desired.

    • Re:Email is mostly broken (Score:5, Insightful)

      by MichaelSmith ( 789609 ) on Monday August 15, 2005 @05:47PM (#13325452) Homepage Journal
      The answer lies in authentication

      And it requires central control. Is this what you want?

    • The problem with these is that they're all duct-tape jobs on the SMTP protocol. The SMTP protocol has fundemental problems in that it essentially has no sender verification and has been configured as much by tradition as anything else to allow MTAs and MUAs to be effective equivalents. To some extent SPF and SenderID try to overcome the verification problems, but at least SPF has serious problems when it comes to forwarding unless header rewriting is done.

      I suppose the "legitimate" spam (not generated by

      • I'm tired of hearing people rant that "We have spam because SMTP is Broken, and SOMEBODY ought to fix it", when they don't really have any better ideas. If you've got any sense of history, you'd remember the complexity of X.400 (which has a lot to do with why almost nobody uses it), and they'd remember the newer UUCP versions that had authentication built in (doesn't stop spam either), and relatively closed systems (market forces either killed them or forced them to interface with Internet mail.)

        The funda

    • I read an article on informationweek.com [slashdot.org] that says spammers are enthusiastically adopting sender id in an attempt to legitimize themselves, or at least avoid filtering.

      But since spamming is legal, those spammers not engaged in phishing or other fraud may choose to accurately identify their mail servers to avoid filtering based on Sender ID compliance. And that seems to be what's happening. Based on a sample of 400,000 spam messages, MX Logic found that 16% had published SPF records.

      So spammers have a 16% ad

      • Gotta use it right (Score:3, Insightful)

        by jfengel ( 409917 )
        If they're adopting SenderID, it makes it easy to filter them. You can't filter just on the existence of SenderID; you need to check who the sender is and ignore email from known spammers.

        That's a good thing. It lets them spew all of the email they want; let's call it freedom of speech (since I don't want any legal limitations on spam also being used to prevent legitimate speech). And I get to ignore them; I can filter them at the SMTP layer even before they get to send the whole message.

        It may not be su
        • This sounds much more effective than the current IP based blacklists which block entire address spans just because of one spammer. Yes, some spammers will have multiple authenticated servers set up, but it will be a lot harder for them to switch quickly. Bot nets will also be incapable of sending spam, which is a major source of spam today.

          • Re:Gotta use it right (Score:3, Insightful)

            by jfengel ( 409917 )
            We'll probably still end up with some IP-based blacklists. You can imagine a spammer who spews out an infinite number of verified IDs. You can't blacklist just the IDs because they're one-shots. Instead, eventually you'll end up saying, "Hey, this server seems perfectly willing to grant IDs to any jackass; let's blacklist the IPs and encourage non-jackasses on that server to get a new one."

            Basically, there will have to be layers of responsibility, and we can encourage the various layers to be responsible
            • No, SenderID tags have to be purchased from Microsoft, and can only be parsed by mail software from Microsoft due to the encumbering XML patents it uses. Take a look at the patent issues surrounding the RFC's for SPF, which Microsoft tried to "embrace and extend" into patented and proprietary uselessness. The current result is that the SenderID keys are not purchased by spammers: they're usually stolen by using the SenderID key's machine as a spam zombie, and it serve the admins of Microsoft mail servers r
        • No, you filter on the existence of a SenderID headers. Its usera are almost entirely spammers. This problem happened before with various bulk email programs that swore up and down they weren't spam, and it has already happened with various header-haiku and other message-ID systems. The technology of SenderID is, in fact, quite stupid. It relies on a Microsoft patented XML header, meaning that you have to waste your cycles accepting the fraudulent email, then processing it, rather than bouncing it on the ba
    • You asked for it, Here It Is. You have officially scored the lowest I have ever personally seen, and I had to actually ADD negative things to the checklist just for you.

      Yes, it's a possibility. Unfortunately, in this case the 'dolts who invariably reply with the survey' are actually right. The survey is funny, but it serves a very important purpose in this case - it shows that completely re-engineering the entire e-mail system means that the problems we have are masked temporarily and then reemerge. Identity, no identity, in the end the 'stopgaps' are actually better than the 'build it from the ground up' solution.

      You Personally advocate a

      (x) technical (x) legislative (x) market-based ( ) vigilante

      approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

      (x) Spammers can easily use it to harvest email addresses
      (x) Mailing lists and other legitimate email uses would be affected
      (x) No one will be able to find the guy or collect the money
      ( ) It is defenseless against brute force attacks
      (x) It will stop spam for two weeks and then we'll be stuck with it
      (x) Users of email will not put up with it
      ( ) Microsoft will not put up with it
      ( ) The police will not put up with it
      (x) Requires too much cooperation from spammers
      (x) Requires immediate total cooperation from everybody at once
      (x) Many email users cannot afford to lose business or alienate potential employers
      (x) Spammers don't care about invalid addresses in their lists
      (x) Anyone could anonymously destroy anyone else's career or business

      Specifically, your plan fails to account for

      ( ) Laws expressly prohibiting it
      (N/A) Lack of centrally controlling authority for email
      (x) Open relays in foreign countries
      ( ) Ease of searching tiny alphanumeric address space of all email addresses
      (x) Asshats
      (x) Jurisdictional problems
      (x) Unpopularity of weird new taxes
      ( ) Public reluctance to accept weird new forms of money
      (x) Huge existing software investment in SMTP
      (x) Susceptibility of protocols other than SMTP to attack
      (x) Willingness of users to install OS patches received by email
      (x) Armies of worm riddled broadband-connected Windows boxes
      (x) Eternal arms race involved in all filtering approaches
      (x) Extreme profitability of spam
      ( ) Joe jobs and/or identity theft
      (x) Technically illiterate politicians
      (x) Extreme stupidity on the part of people who do business with spammers
      (x) Extreme stupidity on the part of people who do business with Microsoft
      (x) Extreme stupidity on the part of people who do business with Yahoo
      (x) Dishonesty on the part of spammers themselves
      ( ) Bandwidth costs that are unaffected by client filtering
      ( ) Outlook

      and the following philosophical objections may also apply:

      (x) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
      (x) Any scheme based on opt-out is unacceptable
      (x) SMTP headers should not be the subject of legislation
      (x) Blacklists suck
      ( ) Whitelists suck
      ( ) We should be able to talk about Viagra without being censored
      (x) Countermeasures should not involve wire fraud or credit card fraud
      (x) Countermeasures should not involve sabotage of public networks
      ( ) Countermeasures must work if phased in gradually
      (x) Sending email should be free
      (x) Why should we have to trust you and your servers?
      (x) Incompatiblity with open source or open source licenses
      ( ) Feel-good measures do nothing to solve the problem
      ( ) Temporary/one-time email addresses are cumbersome
      (x) I don't want the government reading my email
      ( ) Killing them that way is not slow and painful enough

      Furthermore, this is what I think about you:

      ( ) Sorry dude, but I don't think it would work.
      (x) This is a stupid idea, and you're a fascist for suggesting it.
      ( ) Nice try, assh0le! I'm going to find out where you live and burn your house down!

      • what (Score:2)

        by Apotsy ( 84148 )
        What do all those checked items about legislation, politicians, government, "market-based", opt-out, "Sending email should be free", and other nonsense have to do with his post?

        Did you just fill in the list at random?

        • You Personally advocate a
          (x) technical (x) legislative (x) market-based
          You need all three of these to make a system like this work.
          (x) Spammers can easily use it to harvest email addresses - Along with a centralized repository for management comes a centralized directory.

          (x) Mailing lists and other legitimate email uses would be affected - Current e-mail systems would all have to be scrapped.

          (x) No one will be able to find the guy or collect the money - Finding a spammer would be the same as finding a telem

    • The answer lies in authentication - who is sending the email.

      No, the answer isn't authentication. The answer is economics.

      Right now, the recipient pays the primary cost of an email. All the sender has to do is connec to a server, dump some data, and be done. The recipient, on the other hand, has to sort out to whom that data belongs, store it, cache it, pass it on to other systems, drop it in mailboxes, etc. On top of this, the recipient's server must always be online just in case some more mail comes

  • No good publisher (Score:2, Interesting)

    by SW6 ( 140530 )
    It's by "No Starch Press" who seem to churn out books that look good on initial inspection, but don't seem to deliver on content.

    If this was published by O'Reilly, I'd have bought it on sight as they bother to edit their books. As it is, I'll give it a wide berth.

    • Don't know anything about No Starch Press, but I've generally been finding that O'Reilly books need a FAR more critical eye before buying than they used to. I've seen too many lately that need heavy editing, if not a complete rewrite.

  • This should really be entitled "Hiding Spam" (Score:3, Insightful)

    by wernst ( 536414 ) on Monday August 15, 2005 @06:13PM (#13325646) Homepage
    Not to quibble, but even the best filters don't "end" spam.

    Even a manservant reading all of my mail and hand-carying printouts of nothing but personal messages to my Jamacian bungalow doesn't "end" spam.

    It would seem that These Guys [slashdot.org] are actually making an attempt to "end" spam.

    All this guy is just talking about is hiding it from view. Big deal...

    • If you hide enough of it, you end it, because if users don't see it, it stops working. And if it stops working, spammers stop sending it.

      Which raises the question: why do we still get spam? There have been good filters for years, but there is still spam. So it must be getting through somewhere. My guess is that it gets through to (a) people who get email service from their local ISP, and (b) users at medium-sized businesses, who are compelled to use wretched "enterprise" spam filters.

      If everyone used Gma
      • You're making a fundamental mistake here. Spam doesn't succeed because it drives business at 'x' percent. Nobody pays a spammer per hit or per sale anymore--they pay per # of messages sent, regardless of the return.

        We've moved to a market where the product for sale is being sold through a number of venues, and spam is just one of them. Paying someone an extra $1000 to send out a few million emails is no more than insurance of maximum exposure. It might buy you a few sales or it might not--but it's so cheap
  • Blacklist everyone, then whitelist only those people who you really want to communicate with. I've been doing it for years and get ZERO spam. People argue that they will miss important messages - nope, I never have. Email is not the only form of communication. All my family, friends, business clients know how to use the phone if their emails bounce. I have a web form (and phone number) for new clients (and once verified they are whitelisted), and I don't give a shit about the few messages that might no
    • What an absolutely wonderful idea! I'm amazed that no one has though of this yet. Imagine how much benefit this would provide for companies once they had build a whitelist for all "X" thousand employees.

      Seriously, blacklists work on the personal front, if you have a fairly static list of people you keep in touch with. In the business world, it doesn't fly--even if you put the onus for maintaining the list(s) on the users, rather than admins. Business contacts are far too fluid and losing a non-whitelisted m
    • The simplest solution I have found to block spam, one applicable to a larger business scale (but unfortunately not e-commerce), is to, upon reception, store the message in a database 'grey' box and send an automatic 'did you send this mail? Please confirm here' answer if it is the first mail the server has recieved from that address. Upon authentification the sender would be white-listed and can send mail normally. I know this model will only work at perhaps a corporate scale - but could it not be adapted I
    • That works. Doesn't help for people who deal with a lot of mailing lists, and have to allow for offlist replies. Oh, and if you expect me to fill up a web form to communicate with you, you are mistaken.
  • Greylisting solves 95% for me - seriously. Try Postgrey for an easy, built-in solution to use with Postfix - it works like crazy.

  • Some of the previous posters mentioned the rather eccentric views (in my opinion) of the author of Ending Spam (Jonathan Zdziarski). You can sample some of these yourself by reading the essays Mr. Zdziarski has posted on his web site NuclearElephant.com [nuclearelephant.com].

    While someone might have, in practice, unlimited amounts of money, none of us have unlimited amounts of time. So a book is always an investement in both time and, for those with more finite amounts of money, cash. With this in mind, there is the quest

  • How about big fines for the companies that adverstise with spammers? ($1/message!) Figure out how to tax their illegal income and file tax evasion charges! (Works on the mob!)

    Or

    Jhunkhad: A Holy War Against the Infidel Spammers!

    In front of a camera, stand them up and make them recite that they have small, flacid penises and need to refinance their homes and consolidate their debt because they owe all their money to hot horny teen girl web cam sites. Then slap them with a herring until they are unconcious.

    ..

Slashdot Top Deals

He has not acquired a fortune; the fortune has acquired him. -- Bion

Close