Forgot your password?
typodupeerror
News

Interesting Moderation Proposal 263

Posted by michael
from the NP-hard dept.
Kuro5hin is running a story with some interesting ideas for "the perfect moderation system". I'm not sure I care for the overall system but the idea behind it (of 'balancing' out parts of the site with strategic bonuses/penalties) is intriguing.
This discussion has been archived. No new comments can be posted.

Interesting Moderation Proposal

Comments Filter:
  • Slashdot did not set up sid=moderation, it's a unfixed bug in the code that allowed users to start their own sid=anything forums.

    "Unfixed bug" entails that it's supposed to be fixed. Since it is absolutely clear that Rob&Co know about the hidden sids and tolerate them, I'd call this a feature.

    --em

  • by Anonymous Coward
    Hmmm... We need moderation moderators!


    good idea...
    let's call it "metamoderation"
  • by Anonymous Coward
    A good filter would "supress" (without eliminating) the obvious noise and allow low rated quasi-noise. We then decide what ratio of signal to noise we desire or can tolerate.

    The threshold at the top of the page pretty much allows that. You can even change your default by selecting the "save" button. What you describe is pretty much what we already gots. As for filtering out just troll or offtopic moderations that would keep out a lot of legitimate posts that the morons with points mismark, and that does happen a lot.
  • FWIW, it was Fred Brooks in the Mythical Man Month who said "plan to throw one away - you will anyway".

    I think you can build a relatively abuse-resistant system. First, ensure that some *human* effort is needed to submit an article to the website. This means you have to perform some sort of simple on-line Turing test - one example I've seen is "click on the word 'foobar' in this image", but others are possible.

    Second, give voting power only to those who've already gained karma/mojo. Giving any power to an account that has not already gained reputation allows you to acquire power by acquiring accounts.

    Thoughts?
    --
  • What, is the new limit 64? I only lose points because my karma is well over the new karma cap that was enstated. I asked Rob about it, and he told me that it doesn't matter because high karma doesn't matter. Well, he's right, but it would have been nice to mention that on the site.

    The proper place to go to bitch about moderation is here [slashdot.org]; that's one thing slashdot *did* get right.

    Hey, write a better FAQ, and link to it in your .sig; maybe you'll get it on slashdot as a story. You can only expect so much from CmdrTaco and company, and most of the time it doesn't happen. That's why sites like kuro5hin and half-empty exist, because obviously slashdot wasn't enough for some people.

    Personally, I'm still drooling over the skinning support on half-empty. On slashdot, I'd be happy if I could just get a consistent *color scheme*; if Rob implemented a skinning system, I'd probably have a heart attack!
    ---
    pb Reply or e-mail; don't vaguely moderate [ncsu.edu].
  • Well, I don't like the karma cap, but all that means is that you have to keep score yourself. Also, I think that displaying/not displaying karma on your page to other people should be a user-configurable option, because my Karma is definitely part of my user info.

    I think that your other idea is basically sound, but needs some work. Realize that Signal 11, as well as most users that post a lot, get moderated down a fair bit, and not always for valid reasons. Therefore, this shouldn't be a number of downward moderations, but rather a percentage.

    I know (because of the karma cap) that I've gotten 37 downward moderations since it was enstated, but since that only counts downward moderations, that includes a (-1, Overrated) moderation on a (+5, Insightful) post. That is, what would normally be at least a +3 gain in karma ends up being a -1 loss. Under your system, I might be posting at 0 for posting insightful comments; that's why a flat number won't work for everybody.

    Similar proposals have existed in the past, including "aging" of karma. Personally, I would like a simple all-time chart, though, just to keep score. I still think that a really fair way to do moderation probably looks a lot more like kuro5hin, though, and they don't have karma, hence, no need to want to "keep score". That's probably a good thing.
    ---
    pb Reply or e-mail; don't vaguely moderate [ncsu.edu].
  • I've mentions article ratings to Rob before (about 1.5 years ago, about when the current mod system was in place), as only to help guide what the /. userbase wants, but he replied, basically, that he wants to run the site his way, which is completely understandable.

    but with the repeated articles and heavy emphasis on certain topics at certain times, and with the size of /. readership, such as system begs to be put in place. IMO, there's a good way to do this now, similar to kuro5hin, but accounting for the size of /..

    There would be two levels of story moderators: two or three people like Rob, Katz, and Hemos, that can pull a story from the story queue and post it immediately, say on things of dire attention. Then there would be 20 to 30 secondary story moderators, which includes those that already on the list, and others selected from /. readership that are willing to help but understand the /. vision. For any given story in the queue, it needs 1/3 to 1/2 the number of moderators (say 10) "go" votes before it's posted, or the same number of "nay" votes before it's pulled. Thus, not every one of these moderators would see every story, as by the time they got to check on it, it might be toast. Special consideration can be done for duplicated stories in the queue (given the average of 300-400 stories everytime I submit, I'm sure about 25% of those are duplicated), so that by the time the duplicates are weeded out, 30 or so moderators are looking at about 200 posts that might go on /. that day.

    Having a larger number of people would help avoid duplicated stories, excessive focus on a topic, and possibly show a better reflection of the /. userbase as a whole. But again, this is Rob's site, and he's got his right to run it anyway he wants.

  • According to the FAQ on that site, it is pronounced "corrosion" - so chosen because the site owner/editor is "rusty". Kinda inventive, actually.

    --
  • What is a karma freeze?
    __
  • by craw (6958)
    Moderation will always be inherently flawed when moderators achieve moderator powers for arbitrary reasons. I could put a stream of funny posts that get moderated up. Why should I then get the chance to moderate something as +1 insightful. Interesting post seem to mean, sounds good because I don't really understand the post. Interesting should mean something that a knowleable person has not previously realized. Now interesting means is that a newbie just learned something.

    That said, the kuri5hin recommendations is better than the moderation here.

    Question: Why doesn't /. return to the days of the ultimate moderators? What was/were the factors that eliminate this? IIRC, it was because this power was being abused.

  • Heh. Now you see what I gotta put up with every day? :) With this kind of venomous posting and cricisism, after awhile you just stop caring whether you've written something *good* or not and just post whatever crosses your mind. I think that's what really pissed the trolls off with me... even when I don't think I still manage in my worst posts to do better than their best.

    --

  • Some moderators rate articles not on the merits of the content, but on how much it angers them or tickles their fancy.

    Why is this necessarily a problem? Use it to an advantage.. have people sign up to be part of moderator groups. Each group has a specific focus - pro-linux, pro-*BSD, that kind of thing. Be a member of more than one group if you want. The people in charge of that group are in charge of moderation for that group.. if they want everyone moderating, they can do it. If they only want a small group of like-minded people moderating, they can do that too. Form moderator "communities" and let them decide what's good and what's not.

    --

  • Heh. I'm sorry for the rather overly-harsh criticism, michael. I could have put it in a less incendiary way, and wish I had.

    I don't criticize slashdot to be fashionable, I do it when I think slashdot could improve. It's been a while since there's been any improvement here, mainly because of that kind of attitude toward criticism. Dude, think about it, if people care enough to criticise, why are you flaming them?

    No, I don't care at all about upward moderation. I don't moderate here, I hardly ever post here (my user page has one comment showing right now-- well, two by the time you read this), so what does upward moderation get me?

    And, I'd find you comments that misunderstood, but I'm on a modem, and I can't get the story to load before the server times me out, so no dice there. FWIW, the ones that misunderstood were indeed of the dumb, "didn't read the article" variety. No argument there.

    Anyway, I'd have to rate your comment -1, Flamebait. Chill, out, man.

    --
    There is no K5 [kuro5hin.org] cabal.

  • Try this:

    In statistics, it is a common practice to throw out the high and low values, counting them as anomalies.

    If an article gets 3 moderations or more, and at least one of those moderations is different than the others, then pick a +1 moderation to ignore, and also pick a -1 moderation to ignore. Use the remaining moderations to score the article.

    Example:
    Article moderated as follows:
    -1 Flamebait
    +1 Insightful
    +1 Interesting

    This shows an indication that someone was personally offended and mismoderated. Throw out the high and low, leaving a remaining +1 moderation intact. The article gets scored at +1, which is what it would have scored at anyway.

    We want to encourage marking comments UP, so we don't penalize the person who made the +1 score that was thrown out. We WILL penalize the karma of the person who likely made the bad -1Flamebait moderation.

    Furthermore, we need to get rid of the overrated and underrated markers, because they don't stand in meta. I have frequently seen them abused by people who mark comments out of spite in order to dodge the retribution from meta.

    And another thing: Make the +1 score bonus OPTIONAL. Right now if you don't click that little button, the +1 is the default, which seems wrong to me.

  • The posts don't get deleted. If you lose posting privileges you can dig yourself out of a hole by doing some moderation.
  • You are slashdotting my poor linux box! YOU BASTARDS!

    I hope I don't lose my cable service!

    Dammit! I can't take the slashdot effect!

    -The admin of half-empty.org
  • This is a common complaint about the system I've set up.

    The point of the whole thing is that the points will not BLOCK people from posting. There will not be people dominating the site. If there are, there are variables I can tweak.

    The point system is designed for two things
    1) Keep out spam
    2) Provide quirks for users who post consistent, popular posts.

    These quirks include posting in the busiest categories of the day (these change constantly, there are more than 100 categories) and appearing on a top ten list.

    It's an experiment. If it turns out to rely too much on moderation and people are dominating the site, well then I will lighten it up a bit.

    With such an open site, it becomes necessary to implement a system such that spamming can get destroyed easily without my intervention. To do this I have to find a balance.

    Wish me luck :)
  • The points don't censor people. They're designed to keep out garbage. Read my other post on the subject.
  • Note that the moderation system (despite this thread) isn't the purpose of the site. It's also a really neat place and you can post content rich stuff with images and files.

  • I think however you are mistaken with the average/majority rules. For instance, certain people have certain tastes - and because of that you can never have an ideal moderation where you rely on the objective evaluation of people.
    That is the reason with my current project, nerdfarm [currently down, blame network solutions], I decided to throw out moderation on the message boards. Instead of moderation and a rating system - I approach it as, what do most people find interesting/not interesting/annoying, etc. And then try to profile. It's really not that difficult, there are a few categories of rating and then you can just do some simple math off of those.
    This gets rid of trolls because troll posts wont ever be seen unless a certain person pays attention to them.. granted, 5 people may have to look at them before they start being sifted down.
    Also, Karma Whores aren't around -- because the only way for you to score the rough equivalent of karma points is to post what people find interesting and provokes discussion and interaction (the system tracks what users interact with each post)
    Well, in a nut shell that's what I think the perfect moderation system is. However it's just my opinion. If anyone is interested I'll post more details..
  • The moderation on a lot of articles is just plain wrong. Or opinionated.

    The way to fix this is to have the reader decide who's moderation to use. Let moderation be a subjective thing, because the value of a post is subjective. What is +4 to you might be +1 to me. This can be done many ways. A couple of really simple ones would be:

    1. Anyone can moderate as many posts as they want, whenever they want to. Readers explicitly choose which moderator(s)'s scores to use. If I like how Dickbreat moderates things, maybe I'll choose him and let him decide what I see and how I see it. If I don't like how he moderates, I scratch him off my list and choose someone else to moderate for me.
    2. Anyone can moderate as many posts as they want, whenever they want to. When the system tries to figure out what score to show me for a post that I haven't moderated yet, it looks at other people who have a history of moderating similarly to me (using the old 'computer dating' method, for example) and sees what score they gave it. If Dickbreath has a history of scoring articles similar to how I score them, and he marks a post as being +5 insightful, then the system shows that post to me as +5. If I then mark it as -1 troll, then Dickbreath and I start to drift apart and perhaps it don't weight his opinion as heavily, the next time it generates a page for me.
    There are also many variations on this theme. The point is that scores, reputations, and so on, should not be global, but rather, they should be vary by reader.

    BTW, this type of thing works for both posts and articles.


    ---
  • Granted, I'm posting this after there are already 300+ posts, so I doubt anyone will ever see this, but I'll mention it anyways.

    There's a very simple and easy way to have the natural effect of the karma cap w/o actually having one. Once your karma reaches a certain number (say 25, because that's where the +1 kicks in), make it so that things that would give you a +1 karma only have a percentage chance of giving you that +1, and the percentage drops as your karma goes up. Alternatively if you didn't want randomness but didn't mind fractional karma, just award a fractional amount of karma, with that fraction tending to 0 as karma -> infinity.

    What would this do? Suppose we have a poster that tends to average 2 +1s for every -1. Under the uncapped system that poster's karma would go to infinity. Under the cap they'd slam into the cap. Under this proposal, their karma would rise slower and slower until they reach around where they're getting 0.5 for every +1, and would stabilize there. Whatever this number would be, it'd be a lower fixed number than the person who averages 10 +1s for every -1 -- a feature that neither unlimited or capped karma has.

    This does nothing to eliminate karma whoring, but it does make it less and less worthwhile over time. The only way to do anything like that is to change the +1/-1 system itself, and I don't have an answer for that one.

    ---
  • What a perfect example of mis-moderation? Why is this funny? Dragon is making a legitimate, and accurate comment.

    Insightful and interesting? Yes. Funny? No.
  • It's probably too late to post anything here, but let's have a go.

    I'd like to see a web-of-trust/firefly sort of site. Everyone can rate everything, and the similarity of a given user's ratings to your own determines how much impact their other ratings have.

    Everyone would have their own front page, which would be personalized to their taste, based on how stories were rated by people with similar taste.

    Trolls and fanatics would never have much impact, becase multiple accounts would do more harm than good. None of their 20 accounts would be trusted by many people.
  • I have another idea that would help with the moderation problem - simply, for each post moderated there would be some way to say "I like this moderation" or "I think this moderation is wrong".

    If you marked a moderation as being wrong, then after that point the articles YOU read (no one else) would never be moderated up or down by that person again.

    Thus, you have the same set of responses, but a different moderated view for each user. It wouldn't matter how much karma whoring someone did, because you could mark moderators that fell for such tactics as useless. Multiple accounts to moderate from? I can mark moderators as fools faster than they can create accounts and moderate...

    Note that you would never have to know who the moderator was you were disagreeing with, you would simply say that for any post you didn't like the moderation done and to ignore all the moderators of that post from that point forward.
  • I have to agree that part of the S/N problem is pretty inevitable on a system of any size.

    We can improve on moderation though - make Karma relative to posting volume. I'm currently somewhere near the karma cap (was on it, until an IE bug moderated the wrong commented and I rightly slapped down in M2...) but I don't get modded up that often. I'm there as I've been here a long time. And, let's be honest, it's not too difficult to get 50 positive moderations in a year or two.

    Let's say we make karma degrade. Say 1 point lost per month, one lost per 10 posts (or whatever). I need to remain a good poster to keep high karma. Yes, I can see that that _could_ encourage more karma whoring as to keep a +1 you'd need to do so continually, but I genuinely don't think it's as much of a problem as some people make out.

    The other thing to look at is whether it's sensible for posting and moderation karma to remain unified. I'd argue not.

    By keeping them together, we unnecessarily reduce the quality of both.

    Hypothetical situation. Let's say I'm a troll. Does that mean I'd be a bad moderator? No, not necessarily. There's a strong case to be put for not giving trolls mod points as to withold them may deter trolling on some level, but that's about it.

    Now, let's imagine I can't spell and am hopelessly inarticulate. Some might say that was the case now ;) Anyway. I'd be extremely unlikely to ever get modded up for my posts, but that doesn't imply that I wouldn't be a fantastic moderator.

    Ultimately, I'd go for a fundamental overhaul of these things. Rate posts in percentage terms, make sure almost anyone can mod pretty much constantly. Posts then get modded with much more precision - so we don't have the current problem that the first post to hit +5 gets a huge boost over the second, thus encouraging whoring as opposed to well thought-through comments. M2 wouldn't really work so if we want the current style, you do effective M2 by comparing ratings given to posts. Peaks on the chart count as a fair, blips as unfair.

    None of this addresses the fundamental problem with moderation as a concept, though - tyranny of the majority. The comment which works for most will almost certainly win. It may be wrong on careful analysis, but the gut call is that it's correct. Whereas the more thoughtful post with more actual understanding may lose out as less can understand it. Also, what do you do if the trolls win? If the system is entirely automated, it's possible that the trolls could get in on the ground floor and, thanks to all systems being fundamentally based on the approval of peer groups, take over. If they mod each other up and good comments down, you're stuffed...

    If you really want to maintain perceived quality, restrict moderation to a central clique of the great and the good. It's the only way of ensuring that it's good - but it does make the assumption that whoever does the choosing is right, and impose their viewpoint(s) on the group. It takes a very strong individual to be fair and even-handed with that level of power...

  • Everyone here has been brainwashed by the idea of censorship. Moderation schemes are exactly backward, effectively letting others (the moderators) select who and what I read.

    Let me be the only one who filters what I read. Let there be personal, dynamic, intelligent filtering instead of mob-based moderation.

    That is, let my account record moderation that I and only I do to articles. Then, articles are rated (from my perspective only) based on how I've moderated similar aticles in the past. This can be done based on author, keyword matching, whatever -- and should be options in moderation: "I (dont) like this author", "I (dont) like this content", etc.

    Done. The beauty? No text gets explicitly censored -- you read the content that you like, not some function of what the group likes. Of course, you should be able to threshold depending on your mood -- where articles are rated, say, 5=must-read to 0=unclassified to -5=dont-read.

    It's more complicated to code and takes more server CPU, but all side-effects of the current point moderation systems (that everyone is trying to hack and patch around) are gone.
  • I had an email conversation about this exact problem with one of the editors last year sometime. The way I see it, many moderators don't have time to browse at -1 when there are 400 post. So they head to the story that was just submitted and only has 10 post.

    My suggestion was to give the option to have Slashdot serve up a random subset of the stories when I have moderator points. I would then choose to de/promote the ones I was presented with.

    The editors rejected the ideas, saying that it would break up the flow of the conversation, and that the current system works good enough.

  • The problem I've noticed with moderation is that it tends to snuff out alternative ideas, and the people who post them.

    I agree. Furthermore, with metamoderation being able to nuke people's karma scores moderators have little incentive to try to stimulate discussion by promoting interesting, if unconventional or unpopular, ideas. Nearly every time I've done this I've lost karma in m2--people who oppose viscerally a viewpoint apparently have a difficult time swallowing the "interesting" tag I tack onto a post.

    In short, I don't like karma at all. I think it's completely wrong. Each post should be unemcumbered by the quality of previous posts; judged on its own merit.

    If I had more time to read posts I'd probably agree with you more. Karma/moderation helps boost the signal-to-noise to the point that it is sometimes worthwhile reading comments.

    That being said, the moderation process often leaves a very bland set of posts at high moderation levels. As a proposed change, I'd like to see something where I could specify the percentage of posts at a given level that I want to read. E.g., "100% +3 to +5, 50% +2, 20% +1, 10% +0, 0% -1 to -3". That way I can still stumble across interesting "+0" ACs on occasion without having to wade through 100 unfiltered grits posts. Also, it would be nice to be able to "killfile" certain UIDs so when someone is an obvious kwhore (Sig11?) I could choose not to view any of his/her posts anymore.
  • as a moderator, i just finished smoking a large pile of Northstar Glassworks Blue Moon powder, NS-45-p, mixed with some Glass Alchemy Caramel Luster. Mmmmmm!! You gotta use oxy-propane on that, baby!
    . http://northstarglassworks.com [northstarglassworks.com]
    http://glassalchemyarts.com [glassalchemyarts.com]
  • I basically agree that the karma cap serves no useful purpose. Many is the time when I've posted something negative about a story that a popular author posted (e.g. Jon Katz) and burned through 10 karma, just because I dared to speak the truth about the emperor's new clothes. If I didn't tend to have 70-90 points of karma, I wouldn't dare do this. Most of these instances have been where I refuted major points in the story, based on directly observed facts (e.g. that Jon misstated certain author's opinions, when I happen to have talked with a few such people many times and knew this not to be the case).

    That said, there should be no post bonus beyond the basic +1 (and +2 for slashdot superlogins).

    I also agree with the comment that we should be able to rate stories, either +1 point, +0 points (no opinion, for silly people), or -1 point. However, ACs should never be able to rate stories.

    But, I don't think our rating of stories should affect how they show up on slashdot, just provide a table which we can look at (10 most popular, 10 worst stories) and which might give some feedback to those who run slashdot.

  • the idea that some people's posting should be limited seems wrong ... at least here, part of Slashdot's charm is that anyone can post whatever they want, and moderation only helps people filter the list.

    I think I agree. Free speech means that you can speak and whether anybody wants to listen depends on (hopefully the quality of) what you say. A good filter would "supress" (without eliminating) the obvious noise and allow low rated quasi-noise. We then decide what ratio of signal to noise we desire or can tolerate.

    Perhaps I'm concerned I'd be left out...but I'd also miss some of the interesting tangents I read.

    Querry: are we demographicly capable of such "pure" democracy? What is our "troll/hacker" ratio? Regardless it sounds like a truely grand experiment.
  • The problem I see with how many moderators use their points is that they score it according to their opinion rather than the quality of the post. I honestly don't think you can stop this completely but it can be improved.

    My suggestion is to provide two seperate schemes of moderation for each post. Firstly, the moderators score according to the quality of the post (just like it is now) but there is also a separate scale with something like Scores of A to E inclusive which mean A=Agree strongly to E=disagree strongly.

    This could be used to generate a "popularity score" which would complement the existing "quality score" for the benefit of those who want to see it.

    The major benefit of this IMO though is that is encourages the moderator to seperate their feelings about a post from their view of it's quality.
  • As a karma whore turned troll [goatse.cx], let me chime in.

    Why is Slashdot a circus of karma whores and trolls at the moment? Slashdot moderation is failing because of exactly the Delphi effect that Signal 11 tries to promote: the more you spread the ability to vote, the more rating the comment becomes a popularity contest.

    Trolls know this well, and so do karma whores, to an extent: it's a simple matter of politics. Democracy was cracked by power-hungry lobbyists pretty much the same way Slashdot was cracked by the trolls. If you want enlightened comments and worthwhile discussions, then placing the debate in the hands of the silent majority is just going to create a mess.

    Take Big Brother as an example: are you surprised the pretty face with the missing leg won? This shows you how pathetic it was to let the whole Internet vote. If you compare with Survivor, where the people involved in the action did the voting, there's a modicum of sense that arises. The winner was not the prettiest or the most popular, but the one who worked the hardest and made the most sense.

    In Slashdot terms, the problem comes from the fact that those who moderate are necessarely uninvolved in the debate at hand. This allows trolls and karma whores to manipulate the popular opinion, while ACs sink karma faster than an omniscient Battleship player.

    What you want is closer to peer review that you find in scientific papers. You want moderators who know what the hell is going on, and have proved they are unbiased.

    Democracy works when everyone votes. But it doesn't work insofar as empowering everyone with the ability to run the show. Slashdot needs to realize that if they ever hope to run a decent discussion site.

    I'd like to see either Signal 11 or myself get moderated up in a story we don't know shit about, when the moderators are authorities on the subject.

  • Case it point, if I had some moderator points, I'd mod this up. This is a really good idea.

    However, it does have a lot of potential for abuse. Transferring karma across accounts *will* be a problem. Some can mod up their troll comment, and transfer karma from an account that has high karma to one with low karma, essentially avoiding the karma cap.

    Even with its faults its definitely something to consider.

    Maybe these sacrificed karma points wouldn't give a karma bonus to the poster of the comment that is modded, thus avoiding potential abuses. However, it doesn't seem entirely fair to the legitimate comment posters, since he won't benefit from all the karma he might normally have recieved.

    I really like the idea of karma having more of a use than just a +1 bonus.

    Spyky
  • >Slashdot serve up a random subset of the stories
    >when I have moderator points.

    The problem with that idea is the same problem *I* have with meta-moderation.

    When you get a set of posts at random like that you are deneied context. Sure, sometimes a post will quote the parent, but usually they do what I just did: quote only the specific part it's replying to. Rare (and rightfully so) is the post that'll quote the ENTIRE parent before a response.

    Many are the posts that might seem trollish or flameish BY THEMSELVES, that, when you see the parent they are responding to, reveal themselves to actually be sarcastic replies, that, IN CONTEXT, deserve an insightful or intresting.

    Likewise with offtopic. Sometimes a conversation veers off on a tangent that will be quite intresting, but if the post came up out of the contxt of the thread, it'll definately get zapped offtopic.

    This critical problem is why I never meta-moderate. And it's why those asswipes with sigs that say, "moderators: you should be reading at -1, newest posts first", irritate me so.

    I ALWAYS read at -1, modding or no. But mod w/ newist posts first? Fsck no!!! I will bloody well keep reading in nested mode so I can see WHAT a given post is a responce to!

    john

    Resistance is NOT futile!!!

    Haiku:
    I am not a drone.
    Remove the collective if

  • A large user base will increase the noise. There is almost nothing you can do about that. Slashdot's moderation system does a good job of filtering the signal from the noise though. When I browse with a threshold of 3, I rarely see any noise. The noise I see being the karma whores.
  • Here's my thoughts on what to do to fix moderation"
    • Nobody below 10 karma can metamoderate. You have to prove you have some clue before you can judge others.
    • Nobody below 20 karma can moderate. Same argument.

    These two modifications prevent (in large part) the creation of "mule" accounts to alter moderation. Ideally, Rob would also reset everybody's karma to 0 at this point, to prevent the currently existing mules from being able to corrupt the system. (and before you bitch, I'm setting at 125 right now, so I do have a fair amount to lose.)
    • When a user views a page with moderation boxes, encode the target karma into the POST data for the moderation boxes. Don't moderate a post beyond the target karma.
      Example: a post is currently at 2. You are in moderator mode. You select "+1 Insightful". The post data encodes (NOTE! ENCODES! as in encrypted!) the target value of +3 into the data. If the comment has already been moderated up, your moderation is ignored (and you don't lose the point).

    This prevents the common occurrence of a comment being moderated to +5 because several people tried to moderated it from 2 to 3 (for example). I know that several minutes can elapse from the time I get a page to the time I submit the moderation, during which others may have moderated the comment as well.
    • Meta-moderation should use more than 1 vote: any given moderation should be review by at least 3, preferably 5 metamoderators, with a 4-1 majority required to change the moderation.

    Again, this is to prevent mule accounts from corrupting M2.
    • Higher karma individuals should lose more karma for a bad post than lower karma. I say lose 10% of your current karma (round up). Otherwise guys like me can troll for days with no ill effect.
    • Ditch several of the moderation values. I consistantly see posts of nothing but a link being marked "Insightful" WRONG! Interesting, maybe even Informative, but not Insightful. Likewise, I constantly see trolls marked as merely Offtopic. I would reduce the moderation values to:
      1. Interesting (+1)
      2. Troll (-2)
      3. Off-topic (-1)
      4. Me-too (-1)

    • Treat the "high karma +1" bonus as a moderation, and review it like any other. Mark it as "Interesting" so that people with vendettas against high karma individuals won't be able to automatically M2 them down. This make people like Sig11, Bruce (and myself) think about using that point: it's double or nothing - if somebody feels your post wasn't worth that +1, you will lose karma.
    • Also, don't default the high karma +1 to on: make me check the box to use it.
    • Possibly, instead of allowing high karma individuals an unlimited number of higher posts, give a number of tokens per day (say, 1 for every 10 points over 20, round up). Again, this prevents high karma individuals from dominating the conversation.
    • Every day, take the ten accounts that have the most troll moderations. If they have less than 2 such moderations, ignore them. Otherwise, place them on a "Hall of Shame" page. Allow users with more than 20 karma to cast a vote: for or against bitchslapping them. When some number N votes have been cast, if a 2/3 majority is for bitchslapping: SMACK! Welcome to minusville. Otherwise, remove the name from the list.


      This way, Rob&Co cannot be blamed for the bitchslapping: the community did it. This quickly removes the trolls.

    • Keep the AC account. Otherwise, the jackasses will just set up trash accounts. Besides, AC's do serve a useful purpose.
    • Limit people to 1 account per e-mail, and don't allow the use of free e-mail accounts by default (no hotmail.com, no juno.com). If you really do only have a hotmail account, you should ask the SlashCrew for personal intervention, or perhaps a "sponsership" system whereby a person with more than 10 karma vouches for you.

    Now the fuse is lit.
  • Groupthink is exactly the problem. The solution? Disable the ability to take moderation points away from a post. Only allow users to push them up, the trolling/spamming nonsense can be taken care of by a small group of volunteers who will only take down "first post" and "penis bird" posts. Heck, you could write code for that.
    Troll moderation should be eliminated as its been proven many times before that slashdotters can't tell the difference between a true troll and a dissenting opinion.
  • If your comment sucks, we eat you and the poor children of Ireland.
  • "On problem, as with any universal rating scheme is that it would be easy to, say, create 20 accounts and consistantly mark-down a certain author, something you cant do on /. because you would have work the accounts up to moderator status first."

    If you're talking about the system used on Kuro5hin [kuro5hin.org], you should be aware that the number of comments needed to turf or post a story is a percentage. As you create more accounts to affect the vote, more accounts are required to affect the vote. You end up chasing your own tail if you attempt to abuse it that way.
    --
  • So, how come this kind of filtering/moderation system is never discussed? Is it too computationally difficult to implement on a large scale?
    I like this idea, but it runs into a couple of issues:
    1. Scaling issues, because the number of possible correlations between users increases as O(n^2).
    2. Privacy issues, because the system has to keep track of how thousands of users rate things. (This would be a bonanza for market-research people, and perhaps political operatives.)
    If you use a "representative" system and try to group people by a general agreement of taste, the scaling problem is vastly reduced (to O(n*R), where the number of representative profiles R << n). The problem there is that people's tastes drift and differentiate over time, and it might be very difficult to create new representative profiles once the individual ranking information has been obliterated. There is the additional problem of the system being more vulnerable to skewing by proxy mis-moderation, because a representative profile representing a smaller number of viewers is easier to throw off-kilter than one which includes the entire user population.

    I like the idea of trying to restrict moderation to verifiable people as opposed to robots, but I don't see any way to make this feasible that does not involve collecting a lot of information that a discussion site has no business asking for.
    --
    Build a man a fire, and he's warm for a day.

  • Everyone has a point of view that correlates more or less with some others. Let people just moderate things however they wish, and then cluster them according to some reasonable statistical technique.
  • There are my few cents (more than two...)

    1. Advogato's trust metric was mentioned once in the Kuroshin article... it deserves some more looking into.

    2. I like the idea of moderating stories. So do a lot of other people. ./ authors: maybe make a "suggestions" section where people can post about the moderation & meta-moderation system. It may be awhile before there is a system that scales... I don't think the kuroshin idea does... but if we put our heads together, we might find something. Hey, if suggestions are good enough for Denny's, they are good enough for ./

    3. What separates a good moderator from a bad one? A clear head and a good background in the subject of discussion. So, maybe I haven't read the moderator guidelines close enough, so I don't know if it's there, but moderators should be encouraged to only use their points on areas where they consider themselves experts. Maybe this can be enforced by some kind of category management (of course, you could get a thesis in information extraction if you actually pulled this off...) Some similarities here to the kuroshin idea...

    4. No more AC's. Really... what purpose do these serve? Anonymity? Nope, you can just as easily make a bogus account... ./ can track you either way, if they wanted. They just decrease the noise. -OR- alternatively, make it harder to get a normal account (some variant of #1 above) - at least it will keep the stupid posts at 0.

    5. Meta-meta-moderation. Meta-meta-meta-moderation. (meta)^k-moderation. Ok, not really.

    6. Take into account quality in addition to quantity of Karma. If I post 192 times and 2/3 don't get modded and 1/3 get +1 points, and someone else posts 5 times and each gets +4 mod points, who deserves a higher karma? Read my lips: no more whoring :)

    Hmm... I once starting writing a long list of ./ greivances, then I thought to myself... "boy, get a life". I'm thinking that again...
  • In response to how some folks have interpreted this post, I'd just like to say that I was being sarcastic. :oP

  • Give posts +1 for any of the following:
    • "Patents are dumb."
    • "Down with Big Business!"
    • "These lawsuits are absurd."
    • "I think that's a great idea."
    • "Free as in Speech."

    Give posts -1 for any of the following:

    • "Microsoft has done a good job with..."
    • "Maybe this lawsuit is justified because..."
    • "That's a terrible idea because..."
    • "Open Source isn't always the best option..."

  • I suggested something a while back called the flag system. Beside the reply, etc. buttons at the bottom of every article are two other buttons called "Flag Up" and "Flag Down." Flagged posts would draw moderator attention. Some sort of volume system could be used to see what the most active articles are and so maintain the flags meaning (keeping every post from being flagged the equivalent of "urgent"). A sort by flag would be nice I suppose.

    One problem with slashdots moderation system is that it bows to a mob mentality. Napster=Good. MPAA=bad. Please, a monkey could do that. I'm not sure if flagging could help this.

    The other problem is that it is hard to tell what needs to be moderated when there are 300 articles in the comments, you just can't read everything in a reasonable amount of time. Flagging would work towards solving this problem.

  • Hi! We have partially implemented this on a site called Prosebush (http://www.prosebush.com [prosebush.com]). Feel free to make an account (or not) and try to get you post boosted up to the top story. We are presently evaluating the system and would like to have some good data. We haven't implemented the entire algorithm yet for various reasons (I don't think the cheap server we are on would be a good place to do real-time clustering and the C++ compiler is really old). But what we have there is a start. Please try to break it (in the context of the moderation algorithm!), I have no doubt we at Prosebush will learn a lot from it. Thanks for you interest!

    --8<--

  • Think about it-- you cannot create an automated system that isn't eventually susceptible to automated attack. It's that simple.

    I'm going to disagree... It is possible to do algorithmic moderation based on Natural Language Processing (NLP) and an analysis of who read what. I have been yammering about this in several discussions on Kuro5hin but no-one seems to reply so I won't bother reiterating here... but basically you do similarity analysis between what the person has read in the past and the current articles, this lets you know if they will be likely to want to read the current stuff. Further the number of other people who read that article (and it's children) will weight the choice (so a disagreeable, but controversial or important argument, still gets modded up).

    The advantages of such "no hands" moderation include not requiring users to work in order to mod up posts (they just browse), the ability to turn off the rating or weight it in favor of some metric (browse at -1, or just read the Sports news), and articles that show up in a discussion long after the human moderators would have stopped paying attention would still have a good chance of being modded up automatically (for those who come to the dicsussion late but still have good ideas).

    Disadvantages include the fact that it requires a helluva chunk of computer power to do this. We have implemented a more simplistic version of this system over at Prosebush [prosebush.com] and use it to rank story contributions, stories, and genres. So far it seems to work. Please check it and let me know if you think the system is failing to work...



    --8<--

  • Thanks for the links. I'll check it out!

    And to quote Bart Simpson: "I am familiar with the works of Pablo Neruda" (or in this case Bayes!) :) LOL

    Yeah, to have open source stuff for this is ideal... I think there is quite a lot of open source NLP and Neural Net stuff out there but it's probably very academic in nature... I think if you look around University web pages where people do this you might find some packages. I would personally like to see it built into Emacs though! :) M-x moderate-based-on-personal-preferences

    THAT would be sweet! :)

    Thanks again.

    --8<--

  • This is easily fixable. Prevent accounts from transferring more than 2 karma points per day back and forth (or prevent it all together). So you may respond that all it will take is a three (or four) account circle to bypass this. These are quite easily tracked too with a decent algorithym...abusers get kicked out for a month.

    This idea can work. Lets start to build it and lobby Cmdr_Taco for his thoughts on it. It's a much better scheme than the complicated ordeal in the parent story.
  • The factory is the living room, with the television. The product being manufactured... is you.
    Do you have a citation for this .sig? I love it.

    It's the entire narration for an anti-commercial produced by the Media Foundation, which can be found here [adbusters.org].

  • For a regularly updated list of Slashdot's hidden sids, check out Jeremy Driver, aka Prune Whip's SlashList [prunepatch.org] on The Prune Patch.

    There's also a link there for a list that keeps track of the UserIDs belonging to the REAL personalities who post on Slashdot. (Perfect for determining who's a Bruce Perens imposter!)

    MashPotato - Mobile Array of Support Helpers for Potato

  • I wonder how I can get rid of the metamod link.

    I once followed a link that brought me to /metamod.pl but now every now and then I get asked for that page. I'd like to see an option in the preferences "Not interested in metamoderation".

    I am interested in moderation tho.

    M2 just isn't interesting, in the last couple of months I've only "Unfair"-d *one* moderation...
    (yes I do metamoderate, only to get rid of that stupid line on top of the mainpage)

    Ivo

  • From the FAQ [slashdot.org]:
    Why isn't there a link to M2 on my index.pl?
    There seems to be a bug in the Slashcode that prevents the appearance of the M2 link if you've never M2ed. It will also not appear if you aren't logged in. To meta-moderate, log in and head to /metamod.pl After that, the link should appear on the top of your index page.
    Personally, I don't believe for a minute that it's a "bug." I think it's carefully planned that way, and that's not necessarily a bad thing. Also, IIRC, your UID must be in the lower 75% before you can M2 (that's not in the FAQ, it's in the code).
  • The problem I've noticed with moderation is that it tends to snuff out alternative ideas, and the people who post them.

    This leads to a "lowest common denominator" effect on the site in question; a milque toast effect; a trend towards genericity.

    In short, I don't like karma at all. I think it's completely wrong. Each post should be unemcumbered by the quality of previous posts; judged on its own merit.

    Obviously, every site has the right to do whatever they please...
  • I think the need for MM could be reduced if moderators were given real time feedback of the results of other moderators reading concurrently.

    I don't know how many times I've seen a comment that I feel should be rated a 3, mark it up, and by the time I finish reading the other 100+ comments for an article and click the Moderate button, it's now a +5 comment.

    Most likely it's not worthy of a +5 but other moderators working concurrently had also decided to +1 it. I know MM is there to take care of this but I think many of us agree that's not very efficient.

    PROPOSED SOLUTION:
    A Java "real time" moderation interface. It should allow moderators to see "instantly" the results of other moderators.

    Configurable options could include:
    1) The ability to send on your moderation as soon as you make a selection from the pop-up menu as opposed to waiting for a click on the "moderate" button (which would as now send all your mod at once).
    2) If you prefer to wait to the end of your read through of the available comments before submiting your mods, there can be additional feedback in a status bar if a comment you have slated for a mod up (or down) has changed while you were reading other comments.

    While I normally prefer that web sites avoid the use of "heavy" things like Java when ever possible, I think we have a worthwhile use for it here. Of course the conventional method could remain an option too.

    What do others think?

  • Here's an idea for moderation; pay-to-play. Sell moderation points at a dollar a piece, which will cut down on mismoderation. Then, give all the money you make to a good charity. Everyone wins.

    How about it, Taco?
    --
  • Very little of how /.'s moderation and meta-moderation works is documented. How come my karma never goes above 64 even though I get moderated up? Why is it that occasionally it just drops a few points even though I haven't been moderated down? (Does karma age?)

    Once, I asked you-know-who why my karma was stuck at 48. This is a pretty dumb question when you know about karma freezes, which I didn't. He told me to read the source.

    Now, that's all well and good, but it would have taken him less time to tell me what was going on than to tell me to RTFM, or at least the same amount of time. If he had said "karma freeze" I could have drawn my own conclusions and it would have been more helpful than "go try to read someone else's perl code to find out what the system is doing."

    I personally interpreted that answer as "fuck you", and moved on with my life. Not everyone has time to read slashcode, we're too busy posting intelligent, insightful, or funny messages as we karma whore. I'd rather help make slashdot an interesting and successful place than understand how it works.

  • The moderator that moderated this (and I would assume, the at least two others in this discussion) post as offtopic helps prove that the moderation system is really bizarre. (I would guess that this one did it to make a point, but I occasionally read at -1 for a laugh, just because the trolls who complain about 'moderators on crack' are so often right.)

    That said, I don't think this proposed system is all that great ... I wasn't really able to follow it that well, as it seemed quite complicated and I wasn't too inclined to try to sort it all out. However, the idea that some people's posting should be limited seems wrong ... at least here, part of Slashdot's charm is that anyone can post whatever they want, and moderation only helps people filter the list.
  • I think this would be a great idea... perhaps not for Slashdot quite yet, but perhaps in the future when they want to stop employing editors.

    However, I think the real world is this way... moderated by performance of your economic weath and incorporation. Perhaps that is obvious, but I think this article outlines the process of how a voice is heard in the real world. The only thing it doesn't account for neccesarily is the "who-you-know" factor, which could perhaps lie with emailing, and outside-moderation contact.

    Currently I think Slashdot (as far as modeled after the capitalistic process) is much like the current economy. Business are socialized (tax breaks, federal bank loans, etc -- compared to Slashdot's story posters), and the profit is capitalized (companies earn profits the normal capitalistic way, as does slashdot by the traffic of it's posters).

    On the other hand, I think you all might agree that this idea could be insanely great, or just plain suck. Slashdot's strength I feel is it's stability amongst anarchy.

    ----

  • Your algorithm sounds like the easiest thing in the world to break. Remember, the trolls are in general in the upper quartile of readers by intelligence. It's the simplest thing in the world to combine lots of interesting content with a huge amount of incendiary disinformation. On other sites, I've managed to start a huge flamewar on the subject of whether I was a troll or not (I was). I say, bring it on, dude. Implement your algorithm on a site with decent traffic (so that people aren't able to use the "off-line trust metric", the only trollproof one). I betcha that within a couple of weeks, your top rated poster will be me or someone like me, and it'll all be from posting shit.

    Remember the words of the Troll Anthem

    You can't beat the trolls
    You can't beat the trolls
    Ee aye addio
    You can't beat the trolls

  • I worry about this. At the current time, the small community that I steer, MeatballWiki [usemod.com], is very stable, lots of signal. However, as it becomes popular, it will degrade in quality because community doesn't scale [usemod.com] (I fear).

    Now, here's something to think about. The number one security issue is not attacks, trolls and grits lovers. The number one security issue are human mistakes. Most of our security policy revolves around soft security [usemod.com], as it is better to leave community concerns to the community and not to a powerful few [usemod.com]. And over time we will only improve the community with simple systems like KeptPages [usemod.com].

    Unfortunately, an online community is about communication [usemod.com]. It's easier to communicate with 100 people than the 200 000. In fact, you compsci folk already understand the connected graph squaring problem [usemod.com]. Sucks, eh?

    But, I think, I think that the way out is to use the graph density against itself. If you give people the freedom to manage the system, I think a good community structure will emerge. Moderation systems imposed by the site seem disjunctive, not driving the members to solve problems collectively. This is bad. The community must build itself.

    Anyway, as always this is a technology solution [usemod.com] vs. community solution [usemod.com] dilemma. And I'll bet that you're wrong, Signal11. I'll bet that if you let the whole community help itself, there are a lot more people interested in making it work than don't.

  • Not entirely offtopic: what does "Kuro5hin" mean? Is it a hacker version of "Kuroshin" (which also doesn't make much sense)...
  • it seems that it would difficult to ensure that articles are posted that are relevant to the stated subject matter of the site. Slashdot is somewhat Linux/Unix centric. With this moderation system, I do not see how to make sure it does not drift into a sea of noise.

    The Early Days (tm) of the web come to mind, where the community culture was set one way, and then all these new folks (outsiders, etc) got into the act. Not bad really, but you see what happened to usenet. The signal to noise ratio has deteriorated badly into tidal pools of spam, etc. It would be very easy for a message board base on this to go the same route. This seems to be the primary flaw.

    - - - - - - - -
    "Never apply a Star Trek solution to a Babylon 5 problem."

  • The price of fame is that you are popular.

    Now imagine if this was not late sunday night, but some other time that was prone to more traffic, and count your blessings.

    You did say you wanted a stress test, didn't you?

    :P

    - - - - - - - -
    "Never apply a Star Trek solution to a Babylon 5 problem."

  • ``Transferring karma across accounts *will* be a problem.''

    Perhaps it should be "spend 2 karma points to give 1." I've been thinking about it a bit and something this would definitely prevent extensive abuse while remaining "affordable."

    I also like the idea another poster [slashdot.org] ; mentioned about the donor needing to be a good player with karma of at least +20.
  • Limiting on IP address doesn't work, because many are behind firewalls and hence multiple users can legitimately be on one IP.

    this is not a proof that limiting by IP does not work. I think limiting by IP would work fine. If one guy in a company is ruining it for everyone in that company, let those local users solve the problem. If one guy at AOL ruins it for everyone at AOL, who cares? If there are 1000 trolls who've chracked 10 machines each, that's 10,000 IP addresss read-only, with 100s of millions more posting good stuff.

    sounds like a plan, to me.

  • No, it's not a stupid question. You have to visit metamod.pl logged in before you are allowed to M2. This is not documented anywhere that I know of. Sort of a 'hidden feature' of /.

    Where I more cynical I might suggest that this was done so that newbies can't metamod, but I think they've taken away the ability of accounts < 6 months old to M2 recently anyway, so I really don't understand why it's hidden.

  • Metamoderation has it's own criteria about who gets to do it, much like moderation. I think they throw out the last 25% of newly created accounts and also take karma into consideration.

    Normally it appears about once every 24 hours on the main page. If you've never seen it that may be due to a bug in the old slashcode which kept certain people from ever seeing it, you had to discover and go there yourself. The bug also prevented people from being able to meta if they hadn't before a certain point (when they changed to the bug-code? THis is pretty much all fixed now from all reports.) However if you have positive karma and visit metamod.pl, you should be able to metamoderate now. It resets at 7 pm EST so if you just did it, you'll see it come up at the top of your page sometime between then and 8(depending on when apache delivers you the updated content.)

    But the short answer to your question, is that you should see it at the top of your page whenever you are deemed worthy by the little pearl [python.org] scripts that run this site. If it still doesn't come up I'd email Rob and or Cowboyneal and ask them about it, as I'm able to , so by your usernumber so should you be.

    One word of warning, the present system allows you to make changes to the moderators karma for bad moderation, but there is an unusual check in place. If you disagree too often on moderations it will cause you to lose karma. For this reason I reccomend not disagreeing too often no matter what the comments say if your karma concerns you. Two or three bad moderations are about the most you'll be able to deal with per session, so go with ones that irk you the most, and leave any other bad moderations unchecked so the next meta-guy will get them, but for Gad's sake don't approve anything you aren't comfortable with! For other questions about metamoderation try the the semiofficial sid [slashdot.org].

    Fist Prost

    "We're talking about a planet of helpdesks."
  • Here's something for both of you to consider, slashcode already provides the same easy, point and click style moderation for authors as the moderators get, only the authors have no reign on how they are allowed to moderate. I trust *most* of the authors in here to be good folks and only use their downward powers to balance out bad commetns that get modded up, or only moderate down those that truly need it.

    So the question is, why are users allowed to moderate down at all? Sinply take your 5 points and look for funny/insightful stuff. I think OSDN should just go and hire about 5 people to manage all the sites they have that use a moderation system and let them go through and look for true flamebait and abuse. Just a thought.

    Fist Prost

    "We're talking about a planet of helpdesks."
  • Current Moderation is way to complicated, what do others think, less complicated moderation needed?
  • as long as singal 11, and his impressive karma, keep coming up in conversation, i would like to suggest that taco makes an option to have "signal 11's karma" be one of the slash-boxes that you can choose to have on the side of the main page. i think signal 11's karma whoring would make an excellent online spectator sport.
  • I don't know, I get the feeling this would become more of a popularity contest. Rather than ideas getting moderated, people would moderate more on the basis of who posted a certain thing.

    I think there are definite flaws in the current moderation system on /.

    One glaring one I see is that you have to wait forever to be able to meta moderate. This account is at least 4-5 months old and I still can't metamoderate. I even have the +1 bonus available now, and yet cannot metamod, and probably am ineligible to moderate too (I never have gotten any points on this account). While a do agree with not letting new accounts moderate, I think especially think that metamoderation should be available sooner to people with karma.
    -

  • Seriously, there's trolls getting moderated up, and legitimate arguments against Linux getting moderated down, just because the posters are GNU gnay-sayers. (there, that oughta make those bible-thumping moderators happy.) How many times does Bob Abooey have to post "Bababooey to you all !" before he is booted off of Slashdot?
  • I propose the "Akzar model", where we accumalate Karma by fighting for it. For more information see here [cashwars.com]
    Also, there would be a way to "take out" certain users, meaning if they lose enough Karma their posts would be invisible to all. Then they would have to behave for a while in order to re-accumalate Karma.

    That way we could 'punish' lamers and trollers. Good posters could be awarded with cash and/or prizes.

    Just my .02
  • This is primea facea an amazing idea, one thing that occurs to me is that, while it initially seems that the karma in the system remains fixed, if lots of positive moderation is being done this way, modererators are going to do more marking down (if only because all the good comments have been moderated up). This could lead to a 'karma depression', with certain accounts (~11) hoarding karma for the long winters of low karma discontent
    people browsing on +3 will believe that slashdot is broken, as no comments will ever reach such lofty heights. People will start exchanging money for karma, perhaps even karma whoreing in the literal sense.
    Taco might have to start dumping karma on the market, trying to inroduce bouyency, the possibilities are endless.
    Letting people pay to mark down comments would have an even stronger effect, as 2 points would be lost from the system, if this happens, what price a +5 post?

    Am i joking or being serious? I'd like to think both, but i reckon +0, Ridiculous, but cute; would be the perfect moderation.
    &#9786
  • Different people have different tastes in article zposts. Some people like funny posts others like informative. I believe collaborative filtering would work better than a set points system. Everyone votes on the quality of posts and your sorting depends on how closely you mimic others who vote like you.

    CF can be integrated in various ways, but the method I'm thinking of would work similar to this:

    1. Everyone votes on various posts at will. There is no limit to the number of posts you can vote on.

    2. Your votes are compared against other users votes to find the difference in "taste" between you and every other user.

    3. Posts are weighted based on the distance between you and other users. Users who vote like you and vote on posts are weighted higher than users who don't vote like you and vote on posts.

    The underlying math to implement this is rather simple (with the exception of some of the baysian network models) and widely known, but as far as I know, there are only a few very limited open source implementations in existance.
  • by Kris_J (10111) on Monday October 02, 2000 @01:12AM (#741121) Journal
    It depends what you intend to use user moderation for. User moderation will simply reward the current popular ideas. It will penalise anything controversial. If this is what you want, it works great. However, user moderation is not a substitute for editorial control. If Rob wants this site to go in a particular direction, he will need to direct it. If he wishes it to remain a geek-flavoured study into social behaviour then steady-as-she-goes.

    (FWIW I'm currently moving away from collectives such as /. and towards weblog-type things. My favourite source for the unusual and nerd-esq is Penny Arcade [penny-arcade.com]. Sure, it's mostly centered around a comic strip, but the journal entry / news that is attached to each strip is a very good read (even though it's a bit computer game centric). Otherwise I'm returning to traditional news vendors -- I have BBC, Fox, ABC (Oz), Salon & Wired all syncing to my TRGpro.)

  • by IntlHarvester (11985) on Sunday October 01, 2000 @07:59PM (#741122) Journal
    The people who took down the slashdot moderation system did so in an organized and systematic fashion.

    And if you want to see an example of Slashdot just before wide scale moderation and karma went in, check out: Slashdot Moderation Phase 1.1 [slashdot.org]. Even the Anonymous Cowards generally had good comments.

    S/N ratio was pretty high in those days, much like kuro5hin now, but from the look of it, community moderation and a 'reputation system' has failed in it's goal to keep ratio up. Instead we've gotten a Karma Whore 'n' Troll circus with lots of anonymous flamebait on the side, which is entertaining and might generate more pageviews, but is probably not good keeping readership up in the long run. Which is what you'd expect when you let a site 'run itself'.

    Unfortunately, it's probably only a matter of time before kuro5hin is 'hacked' and it's system has been rendered as useless as Slashdot's for encouraging valuable posts. You are damned if you do (let the readership moderate itself) and damned if you don't (admins end up censoring content, people rebel against 'elite' moderators.)
    --
  • by Ratface (21117) on Monday October 02, 2000 @12:51AM (#741123) Homepage Journal
    You might want to start looking up some stuff about Autonomy [autonomy.com]. This is possibly the fastest growing tech company in the UK. They have a product that uses NLP to help with search solutions, agents, and knowledge management (to use the new-media-hype terms).

    I have seen their products in action and they are *extremely* impressive. What might be of most interest to you is the way their algorithms work. Now I'm sure they won't release that info, but you may be able to glean enough info from their material to get on the right track. (Start here [autonomy.com] perhaps). Basically their technology is based on Bayesian algorithms (Bayes was an 18th century English cleric who came up with some cool ideas about NLP that couldn't be proven until recently when the computing power and information volume to make them work became available - how's *that* for far-sighted!) combined with Shannon's Information Theory. It is WAY powerful in practise!

    I agree that it would be really cool to see some kind of automated moderation system based on this type of principle. I'd also *love* to see some Open Bayesian NLP work.

    Good luck with the experiment.

    "Give the anarchist a cigarette"
  • by tomreagan (24487) on Sunday October 01, 2000 @07:21PM (#741124)
    How can there be a perfect system for moderation? Moderation is just that - a moderation of thoughts and desires, really nothing more than a system to balance the will of the masses against the will of the few. That is, letting people read what they want and ignore what they don't vs. being forced to read what they don't want and listen to what they don't like. Too little moderation and the little voices are drowned out by the big guys, too much and the same thing happens. So in that sense, it's really just a great big continuum.

    I think that in the end, moderation is really just a personal preference issue and not worth that much effort. None is boring, some helps, but too much is worse. Though I must applaud the efforts going into this, and I do enjoy the debates the issue provokes, maybe instead of looking at it like and trying to build the "perfect" system, we might be content to just call it exploring options or do something more interested. But trying to "perfect" moderation seems silly.
  • by WillAffleck (42386) on Sunday October 01, 2000 @10:48PM (#741125)
    It would be a good idea to allow someone to buy moderation, where:
    1. the donor has more than 20 karma (e.g. is a good player);
    2. the donor can push up a post only to a total max per post that levels it out at 2 points (e.g. nasty people mod down a good post to -5, nice donors can only spend it up to +2, cost of +7);
    3. extra karma spent to buy moderation costs 10 karma for every karma given (e.g. +7 would cost +70 karma);
    4. each donor could only spend 10 karma per story (e.g. you'd need to find 7 people to back you up in my example, and they would not be able to donate to any other post in that story);
    5. a donor can only donate up to 10 karma once per day (to keep karma whores from abusing it).

    Concept?
  • by JohnDB (51703) on Sunday October 01, 2000 @07:34PM (#741126)
    How About this:
    1) If you think you were unfairly moderated, challenge your moderator to a duel (Say... High noon in front of the Geek compound)

    2) You beat each other until somepody passes out.

    3) The loser has his picture taken in all his bloodied glory, and posted in a new 'moderation' photo gallery for all to see.

    Maybe that will get the moderators to be a little less biased, having to worry about their own butt.
    ...Then again, that'd probably be illegal in most areas of the world.

    jdb
  • by jonnythan (79727) on Sunday October 01, 2000 @07:29PM (#741127) Homepage
    In all of the moderation systems I have seen proposed, there's something missing that keeps all moderation systems ridiculously stupid. It's called negative feedback.

    All of the systems I have seen either totally lack feedback at all (for instance, Slashdot's miserable attempt called MM) or have some type of positive feedback - this proposal included. I may be wrong, but it seems that a "poster" with an "idea" can grow to wield power. Someone accumulates enough points, he or she can consistently post to the most active categories and be seen. What would this lead to? A few people who are very active and dominate the board.

    I don't know what to propose as a negative feedback system for this type of situation. All I know is that I see a desperate need for someone to design a system in which all users share power equally, however cheesy that may sound. I would love to see a proposal that intends to keep power out of a select few individuals so we rarely see the same name twice. Everyone could have the power to moderate every story they see. The bad stories will quickly drop off the front page, and the good ones will stay. The site won't have the highest-quality news, but you simply can't have both.

    What you will get from a site like this is a myriad of opinions and views from all sorts of places. It won't be CNN, and it won't be nearly as homogenized as /.'s "napster, mp3.com, linux" current portfolio. This kind of site would be a great place to get a fresh look on things.

    This is just an idea, but such a site would cater to the crowd that kiro5hin has attracted. Hence, I think it would provide a nice counterpoint to Slashdot, where one can get a more mainstream, less off-the-wall and imaginitive, quality news source.

    Again, admittedly, I didn't inspect the original story too closely..I read the first half and skimmed the second. Just felt like giving something to think about though.
  • by mOdQuArK! (87332) on Monday October 02, 2000 @08:56AM (#741128)
    Every time this subject comes up, I see a lot of people arguing about variations on a global moderation system - I keep wondering why nobody brings up the possibility of collaborative filtering.

    Basic idea: allow everyone to mark whether or not they like particular articles and/or authors, then use that profile for each user to find the articles and/or authors that user is probably more interested in.

    This is even more useful when you compare user's profiles against each other so that if two users have been marking messages/authors in such a way that their profiles are very similar, then when one of the users marks a message/author favorably, it automatically gets a higher rating for the other user (or down, if marked unfavorably).

    In effect, each user will statistically get messages rated through a moderation scheme based on their own past history, and the history of other users who happen to agree with them.

    So, how come this kind of filtering/moderation system is never discussed? Is it too computationally difficult to implement on a large scale?
  • by yuriwho (103805) on Sunday October 01, 2000 @08:05PM (#741129)
    Cheers to this idea!

    Even if it is just an added option to the current system. To make this transparent the spender/s should be identified!?!!!! That would be really interesting and would prevent siggy from dominating moderation for the first couple of weeks.

    Just think...do I spend on this comment and lose my +1 for the good of humanity? If it was done transparently this would be even more interesting as your moderations would tell as much about a person as their posts.

    I say lets do it.

  • by Carnage4Life (106069) on Sunday October 01, 2000 @09:04PM (#741130) Homepage Journal
    The problem is not moderation in any shape or form but instead human nature. It doesn't matter if there are a few moderators a la slashdot or a lot a la kuro5hin, people will generally appreciate an opinion that reflects theirs and be hostile to an opinion that is in conflict with theirs. If there was an open way of holding people accountable for their moderations, just as we can read vote histories in article submissions on kuro5hin, then maybe people would be more careful with their moderations (or it could become like slashdot where the many tyrannize the few via meta-moderation).

    In my opinion all moderation is flawed because it relies on human nature which is inherrently flawed. I personally suggest reading without scores, after all USENET has no scores and this did not alter the quality of the discussions in several groups.

    Second Law of Blissful Ignorance
  • by nomadic (141991) <nomadicworld@@@gmail...com> on Sunday October 01, 2000 @07:25PM (#741131) Homepage
    Why else would they be home all day long to claim "f1rst p0st fuxxxxerz!!!JH(*)&# eye 0wn jew" and talk about naked and petrified actresses?

    I always thought it was some form of performance art.
    --
  • by gunner800 (142959) on Sunday October 01, 2000 @08:22PM (#741132) Homepage
    I say we stick to the conventional wisdom of "Everything in Moderation". Moderate everything.

    If you don't like a post, just click "-1, don't like it" and be done with it. Don't like a story, moderate the story as "-1, post-IPO-esque" and you're through.

    Strong opinion about a particular user? Moderate every he has posted and ever will post. Just think, with out combined might we can hand out a bitchslap even the Taco would envy.

    Heck, you even moderate moderation. If you disagree, there's "-1, wrong". If you like it, it's "+0, right" (because if its right, it doesn't need to be moderated). Hell, you can moderate the whole moderation system; if it gets low enough, it will be defuncted and Anarchy shall reign.

    Moderate Slashdot as a whole...negative moderation will bring more stories confiscated from Kuro5hin, positive yields more Jon Katz stuff.

    It's extends to the micro-level as well. Moderate the topic list, color scheme, each others' passwords...

    Moderation at the bit-level may be difficult, until we get quantum computers and get moderate an individual bit as "+5, very" rather that just "0, false" or "+1, true".


    My mom is not a Karma whore!

  • by DickBreath (207180) on Sunday October 01, 2000 @06:59PM (#741133) Homepage
    Something needs to be done.

    The moderation on a lot of articles is just plain wrong. Or opinionated. Some moderators rate articles not on the merits of the content, but on how much it angers them or tickles their fancy. Articles that follow the groupthink of the cult get good moderation.

  • by tolan's my name (234431) on Sunday October 01, 2000 @07:15PM (#741134) Journal
    On problem, as with any universal rating scheme is that it would be easy to, say, create 20 accounts and consistantly mark-down a certain author, something you cant do on /. because you would have work the accounts up to moderator status first.
    On my reading of the summary the system doesnt seem to have a way to deal with this. Indeed it doesnt really seem to be adaptable to deal with this.

    On a positive note offensive comments should go down quicker than they do on /., and the system allows for a finer grading of articles, but i cant really see it being better in the end.

    Since this is going to lead to an inevitable discusion about /.s moderation scheme my two-pennies worth is that you could combine this scheme with /.s in a 3 tier manner. Any reader can vote a story up or down, however no effect registers until a moderator (generted in the /. manner) comes online. They are then presented with the top x movers, and check to see if the public vote is accetable. They can then ratify or veto the decision. It could even take 2, or 3 agreeing modrators before the story was moved up or down. Metamoderation would then take place in a similiar way to at present.

    However i dont really think that the system would be an improvement on the present one, which, given the circumstances, works rather well (though i might only be saying that because i got +4 karma today &#9786).

  • by Signal 11 (7608) on Sunday October 01, 2000 @07:26PM (#741135)
    Okay, as the resident Karma Whore and de facto expert on moderation here on slashdot, I think I'm probably the one person here who can comment with authority on this article.

    My gut feeling - it won't work. I think the ideal moderation system would be based on what is called the Delphi Effect, if I understand it correctly. Basically the more people you have moderating, the closer to the "true" rating it will be. ie: if 80% of people believe a post is +3, but 20% believe it is -1, it is +3. No averages. A sort of majority rules. It depends, however, on alot of people moderating.. to the point that there are more moderations than there are posts. But, my ideas aside...

    The problem with this proposal is two-fold: One, it has no way to detect 1 person using 10, or a hundred, or a hundred thousand, accounts and thus biasing the voting system. It's a problem prevalent here on slashdot where the trolls have created throwaway accounts. Limiting on IP address doesn't work, because many are behind firewalls and hence multiple users can legitimately be on one IP. one account per e-mail address doesn't work either - e-mail addresses are easy to get.. often for free. The net result is a small group of determined attackers can destroy the system (sound familiar?).

    The second problem is related to the first. Their idea of having the users rate themselves initially is a very good idea (rob, you paying attention?) but it suffers from the fact that someone can simply moderate their own posts, and gain a point advantage.. and we're right back where we started.

    The key to moderation lies in accountability. You can create the best system in the world - but unless you can enforce some kind of "one person, one vote" standard, it will always be open to abuse.

    Lastly, some advice for the kiro5hin maintainers - don't count on obscuring the statistical system to deter your attackers for long. The people who took down the slashdot moderation system did so in an organized and systematic fashion. These people are bored and have nothing else to do - but you DO and hence are at a disadvantage. Once the system is in place, PLEASE ADAPT IT - don't just deploy it and forget about it. It'll need to be tweaked, updated, maybe even entirely thrown out for a new system. Trust the wisdom of Strousoup(sp?) on this one - design the first implimentation to throw it away, you're going to do it atleast once anyway.

    Cheers,

    Signal 11

    --

  • by The_Messenger (110966) on Sunday October 01, 2000 @07:31PM (#741136) Homepage Journal
    ... I think Slashdot needs to remove the +50 KarmaKap. We all know it's not supposed to be about the karma, but why bother encouraging quality posts with a reward system if you can only go so high?

    Some of you may feel that without such a cap, it becomes a game (as in "karma whoring" AKA "Signal 11's life"), but I think that's irrelevent. The only "downside" to this "game" are more interesting, informative, and funny posts -- not much of a downside, eh? Some call it "karma whoring", but I think that it's simply the knowledge of how to be a good Slashdot citizen.

    Yes, I check my karma totals, but not because I feel it's a game. I use it to gauge the overall effect my contributions to the site by observing the rate of karma growth. And if my karma drops suddenly one week, I know it's time to cut back on the flames and trolls. ;-)

    And yes, it's also fun. I'll admit that.

    I understand that one of the major reasons for having such a cap is to avoid abuses like this: let's say Signal 11 post interesting, insightful, and funny things for a year. From my experience, an active, consistently good poster can, on average, easily get 100-200 in a year. This means that Signal 11 could start posting Goatse.cx links, and it'd be weeks before he loses his +1 bonus.

    I have a proposal to solve this: if a user is moderated down more than 20 times in one week, he loses his +1 bonus for a week. If he is modded down more than 30 times a week, he posts at 0 for week, then at 1 for a week. If he's modded down more than 40 times a month, he posts at -1 for a week, 0 for a week, and 1 for two weeks. I think this not unreasonable, considering that most users with enough karma to get the +1 bonus won't suddenly become PBG or *syringe. :-)

    Who else agrees that the +50 KarmaKap has to go?

    ---------///----------
    All generalizations are false.

  • by kuro5hin (8501) on Sunday October 01, 2000 @09:49PM (#741137) Homepage
    There's been a lot of misinterpretation of this article, due mostly to the useless writeup at the top of it. Thanks for the link, but when did Slashdot stories stop providing *any* useful information about the link?

    Anyway, the mod system described is not the one on K5. It's in Glasscode, which is the system the article is about. Which does not run K5. (Note: I'm not sure if you understood this or not, Sig11, but a lot of other people got it completely wrong, so that's for them).

    Lastly, some advice for the kiro5hin maintainers - don't count on obscuring the statistical system to deter your attackers for long.

    Which part of the statistical system do we obscure? If you want a full and detailed explanation of how the various K5 systems work, see our relaunch article [kuro5hin.org].

    Otherwise, I agree-- the system always has to be evolving. Think about it-- you cannot create an automated system that isn't eventually susceptible to automated attack. It's that simple. You just have to make it hard enough to attack that it's not worth it, and use the lag time to keep ahead of the kiddys. And sooner or later, you lose the race, pick your ass up, and try again (c.f. this summer for K5).

    --
    There is no K5 [kuro5hin.org] cabal.

  • by Frac (27516) on Sunday October 01, 2000 @07:46PM (#741138)
    I wish I can moderate articles on slashdot. In fact, I wish there are little radio buttons next to each article that says:

    I don't want to see this article again and again

    or

    Please post this article again and again and again

    so CmdrTaco and his very diligent team will know which articles we want to see redundantly.

    Slashdot - News for Attention Deficit Disorder. Stuff you saw yesterday.

    On a slightly more serious note - surely Slashdot must be getting very uninteresting for the Slashdot admins if they arent' reading their own site. What does that say about quality control?

  • by devphil (51341) on Sunday October 01, 2000 @07:52PM (#741139) Homepage

    Very little of how /.'s moderation and meta-moderation works is documented. How come my karma never goes above 64 even though I get moderated up? Why is it that occasionally it just drops a few points even though I haven't been moderated down? (Does karma age?)

    Why can't we talk about moderation somewhere on Slashdot? If it gets brought up in a normal discussion, it's -1, Offtopic. I've never tried to submit a Slashdot article that concerns Slashdot itself, but the people who have say those are rejected.

    How about a new category: -1, Herdthink, for those posters who just spew the party line about "and this is why Linux is so much better." At the very least they shouldn't be getting Insightful points for copy'n'pasting stuff from the FSF or OSI's webpage verbatim.

    If we had a better FAQ, it would at least contribute to more "Insightful, Interesting, and Informative" discussion about moderation.

    Enh, just my two timeslices.

  • by DeadSea (69598) on Monday October 02, 2000 @01:12AM (#741140) Homepage Journal
    I most often see things I would like to moderate like this in older articles. It seems that moderators never hang out in anything that is no longer near the top of the front page. Here is my analysis of why it is happening and what can be done about it:

    The first problem I see is that the first posts to an article are the most likely to be modded up. Moderators tend to hang around the couple most recently posted stories.

    Not everybody refreshes Slashdot every two hours. The people that do, are the ones that agree most with the stereotypical slashdot agenda.

    Insightful posts take time. It could easily take an hour to *read the article*, do some other research, and post some meaningful commentary. Those who post fast seem more likely to spout out their gut feelings.

    To sum up: The people who post first are likely to be avid slashdot readers and more zealotous. Posts that are made soon after the article goes up are not as likely to be based on facts.

    On hot trigger issues such as this one, I have read comments soon, then comments later and been pleasantly suprised by a couple better posts that get moderated later. Often on looking further, I notice that there are several more that I would have modded higher than the ones that are modded higher.

    Let me try to illustrate this with a graph:

    PostQualityv sTime:
    |high
    |
    | +---+
    | +++--+
    |+-+ +---+
    |++ +---+
    |++ +---------------
    |++
    |+-+
    |++
    |++
    |
    |low
    +-----------------------------------------
    time-- ->

    Sumofmoderationdone
    |more +-------------------------
    |&nbsp ;+-----+
    |+--+
    |++
    |+
    |++
    |+
    |+
    |+
    |+
    ||
    |
    |less
    +-----------------------------------------
    time--->

    As you can see from the graphs, I think there are a lot of good comments posted later that don't get moderated, while a lot of earlier comments that might not be quite so good, do.

    I suggest the golden moderation system.

    You get 5 moderator points.
    2 of the are gold.
    2 of them are silver.
    1 of them is bronze.

    gold points can be used on any post at any time. Silver points can be used on posts attached to articles that are more than 2 hours old. Bronze points can be used on posts attached to stories that are more than 1 day old.

    I think this would really do wonders for Slashdo

  • by aardvarkjoe (156801) on Sunday October 01, 2000 @08:08PM (#741141)
    All too often, there are posts that really need a +1 or -1, but the choices just don't cover it. Therefore, I say we need the following choices added:

    +1, Troll
    /* Really, we need more than just 'Funny' to reward the good ones. */
    +1, Whore
    +5, Signal 11
    /* So we don't have to waste multiple moderators' time. */
    +1, Slashdot Already Posted This
    +1, Claimed They Were Expecting -1
    +1, Redundant
    /* Perhaps split into several categories, such as 'about [copyrights | patents | free speech | privacy | gun control | jon katz]' */
    +1, Only Intelligent Post in the Entire Discussion
    -1, Stupid
    -1, Clueless
    /* I expect these to be used rather heavily */
    -1, Opposing view
    -1, goatse.cx
    +1, goatse.cx

    There could be many more, of course ... Perhaps, instead of the list, we could have a textbox where moderators type the reason for their moderation.
  • by \\x/hite \\/ampire (185046) on Sunday October 01, 2000 @07:30PM (#741142)
    How many times have you seen a post that really really deserved moderation but couldn't do a thing because you weren't privileged enough to be a moderator at the time? Many times I'm sure. How about this... in addition to the usual "randomly chosen" moderator stuff, for each story a user would be allowed up to three moderations. The catch, each of these would be at the cost of one karma point each. Why would this work?

    You need to be logged in to moderate

    You must have karma to give karma

    It would actually give some type of value to karma

    Honestly, the only way I can see to abuse such a system would be transferring karma across account, but why would anyone really need to do that? ;-)

The test of intelligent tinkering is to save all the parts. -- Aldo Leopold

Working...