You are viewing a single comment's thread.

view the rest of the comments →

2
58

[–] Amarok 2 points 58 points (+60|-2) ago  (edited ago)

@Atko, we've talked about this before. There is a solution but it is fucking hard to implement. Reddit will probably never do it, the change would be too disruptive to their codebase. You, however, are in a position to experiment in ways that reddit will never be. I suggest you take advantage of that. Here's a complete and updated breakdown of this idea and one other idea for sorting that would also help with moderation. I wrote this up for reddit and haven't got time to rewrite it right now, so mentally in your head just substitute subverse and voat into the text.


On The End Of Eternal September

This idea is summed up perfectly in a single word: Seniority.


Ask yourself this... why does a sub with 1000 subscribers self-moderate near-flawlessly using nothing but the most basic up/down voting system... while a sub with over 250k subscribers inevitably becomes a cesspool of group infighting, recycled content, and lowest common denominator trash? Why can't we maintain the quality of the original, smaller sub as it grows without resorting to power moderators, complex rules, and bots - which eventually create their own problems?


Well, what's the real difference between these two places?

When the sub starts, it is invisible, only attracting subscribers that actively search it out. The people subscribing in the youth of the sub are the people with a direct interest in that sub's topic matter and the intelligence to find it. The sub develops culture and a common behavior due to these early adopters. They make the place interesting enough to attract more users. This core set of users has far more good actors than bad actors.

As the sub grows, however... (1k to 250k)

  • the rate of submissions rises quickly (5 a day vs 200 a day)
  • more new users (who don't know or care about the culture) find it more and more easily
  • the ratio of moderators to subscribers goes way out of balance (500 to 1 vs 125,000 to 1)
  • old mods are tired, vastly more mod work is required leading to burnout or laziness
  • new mods are less keen on the subject than old mods who created the place, good mods hard to find
  • cliques of users formed that want different things, fighting with each other over what's on topic
  • spammers are drawn in by the popularity and begin aggressively promoting crap content
  • bursts of new users and drive-by voting are common due to high profile submissions
  • people begin sharing links to the subreddit in conversations all over reddit causing more of the same
  • content quality slides, driving away more discerning contributors and original members
  • original culture is lost, alternative subreddits spring up as older members move elsewhere
  • these smaller communities are better content stewards than the larger, older subreddit
  • leading to this cycle repeating itself over, and over, and over

We've all seen this play out on reddit a couple thousand times. Frankly, I don't think this problem can be solved by traditional human moderation. If we don't try something new, then this situation is not going to improve. Basic up/down voting has conclusively proven itself inadequate to the task of large scale moderation.


We usually turn to large moderation teams at this point.

This can go either way, we've seen spectacularly good teams and spectacularly bad teams on reddit with most falling in the middle. The original creators don't necessarily know how or want to run a large team. Power moderators end up on many teams causing concerns about the website's integrity. I know of precious few moderation teams that have managed to grow large and maintain their quality, and the few that did, tend to have subs that have an incredibly narrow topic focus and employ heavy handed censorship.

The solution is to find a way to somehow, organically, turn those original 5000 people into pseudo-moderators by the time that crowd of 250k people has shown up. This is a different kind of moderation, where the original members quietly acquire distributed power. The key question here is, how do we grant them greater power, and how do we keep it distributed fairly, with power coming from the aggregate in a democratic fashion, rather than from a chosen few?


In the end, this is all about identifying "good faith actors" in your community and handing them a small increase in power. The hard part is getting that power in the hands of the right people. Who are the right people? There are a few good bets we can place...

  • early subscribers who were there when the place was young and helped build it
  • people who've earned a lot of karma from within the sub itself, both link and comment
  • 'approved submitter' types, often distinguished by flair for their contributions (like askscience experts)
  • people who are actively visiting the place and voting often, even if they don't comment (hardcore lurkers)
  • generally, older users are more likely to know the rules and the culture than new arrivals
  • special 'calibration posts' can be used to identify good faith actors, explained further below
  • perhaps look at how many votes a user has cast within that subreddit as another metric

So now that we have a way to identify the good faith actors, how do we give them the power?

It's incredibly simple to implement, really... all you have to do is make sure that their votes count more than everyone else.

Know that this is playing with fire. Make their votes too heavy, and you will create power user cliques and turn every subreddit into an echo chamber that buries new ideas. That was part of digg's problem. Make the votes too light, and you'll end up with reddit's problem, where all uniqueness and quality is washed away by a sea of lazy new users crashing the party. This is why so many defaults 'go to shit' and you see users saying that subs with more than 10k people aren't worth visiting. They aren't wrong, this is a very real effect and we need to solve this problem.

There must be a balance between these two extremes. On striking that balance, the voting system itself should be much better at moderating content. Normal moderators become people who merely need tend to the CSS, or set a few dials and fiddles on subreddit options. The mods won't need to police content as rabidly as they do now because thousands of subscribers that have been automatically identified as acting in the sub's interests are empowered to do that job.

I think the number one most important thing we can do for the long term health of this website is to experiment until we find that balance. This is as democratic as moderation can be, drafting thousands of original subscribers to become the moderation team.


I have to continue this in the reply, apparently it's too long. :P

0
41

[–] Amarok 0 points 41 points (+41|-0) ago 


There are five critical points I want to make about any implementation of this idea.

First, this system must act uniquely in each sub, rather than as one large sitewide system. Let every sub run their own experiments, and we will quickly find out what works and what doesn't. Mods are free to leave this system off as well - I don't expect most to feel the need to use it until they get past 50k subscribers.

Second, the differences in the weights must be small to safeguard against creating a power user clique. If someone has a weight of 10, that's giving them ten times the power of a new user which is very extreme. I'm not convinced that is a wise choice, though I could be wrong - only by experimenting will we know for sure. We should start small and if necessary move it up slowly until we hit the right balance. I'd start at two or three points maximum. I suspect larger, older subs might need higher totals than newer, smaller subs to achieve the same effect. Something like the sub's age in years might be a scaling cap, max 3 points for a 3 year old sub, max 7 points for a 7 year old sub - or base it on subscriber numbers.

Third, we need to see both the total number of votes cast and the total weight of those votes as separate numbers. We don't want to lose sight of how many real people voted on something, and we need to see the weighted total to see if this system is having an effect, and if that effect is positive or negative. This is also to help with transparency. Perhaps only the moderators get to see the weights and those aren't made public to the users to avoid people gaming the system - or perhaps the mods can choose to make that information visible to the users if they so desire.

Fourth, and arguably most important - new submissions all start at the same weight, regardless of who is submitting them. We are not trying to create power submitters, we want empowered voters. If someone's submission starts at 3 points vs a new user's 1 point we will end up recreating digg, where power submitters dominate the content. We don't want to go there. At least if they all start at the same weight, it takes a minimum of two people to bump something. Reddit already does some detection of and compensation for vote manipulation, this protection needs to be applied here as well.

Fifth, these weights only exist on and only affect submissions and comments (submissions being the most important of the two). One's karma is still calculated at one point per person per vote, no weights, exactly as it is now, and I am not in favor of ever changing that. Weights are for helping the ranking algorithms to moderate content and nothing more.

Seem like reasonable, safe limitations for a starting point?


Now, let's get specific. I'm going to use a large/default sub as an example.

Imagine in the mod control panel, we see something like this...

  • [ enable ] vote weights
  • cap vote weight totals at a maximum of [ 5 ] points for this subreddit
  • user has been subscribed more than [ 365 ] days, add [ 1 ] points to vote weight
  • user has been subscribed more than [ 730 ] days, add [ 2 ] points to vote weight
  • user has been subscribed more than [ 995 ] days, add [ 3 ] points to vote weight
  • user has acquired more than [ 100 ] comment karma from this sub, add [ 0.5 ] points to vote weight
  • user has acquired more than [ 1000 ] comment karma from this sub, add [ 1.5 ] points to vote weight
  • user has acquired more than [ 100 ] link karma from this sub, add [ 0.5 ] points to vote weight
  • user has acquired more than [ 1000 ] link karma from this sub, add [ 1.5 ] points to vote weight
  • user has acquired more than [ 100 ] karma from sister subs [ list ], add [ 0.5 ] points to vote weight
  • user has acquired more than [ 1000 ] karma from sister subs [ list ], add [ 1 ] points to vote weight
  • user has voted well on calibration posts, add [ 1 ] points to vote weight (explained below)
  • user has voted poorly on calibration posts, subtract [ 3 ] points from vote weight (explained below)

No matter what happens, the max here is 5 points, the minimum is 1 point. The user's actual weight is calculated based on the other metrics adding or subtracting from his total. Anything within [ brackets ] in those examples is set by the moderators however they want it to be set.

Here's what we're trying to achieve with this system.

  • around 10,000 top contributors, oldest subscribers, and moderators have effectively a 5 point vote
  • next tier of 100,000 users are mature subscribers and regular contributors, between a 3-4 point vote
  • next tier of 1,000,000 users have been around long enough to learn the ropes, 2-3 point vote
  • remaining 2,000,000 users aren't even a year old yet, between 1-2 point vote
  • the 150,000 new subscribers this month have a 1 point vote at best
  • bans now have real teeth, earning one's way back in is not easy and takes time
  • brigading voters have a 1 point vote since they don't subscribe or contribute, blunting brigades
  • spammers are less effective at bumping content using new accounts

In theory this will result in more on-topic and good faith moderation of content using the voting system.


The drawback, of course, is that the algorithms for the front page and /all and any multis now have a dramatically harder time accounting for the ranking of the content when they compare subreddits with different weights and subreddits with no weights. If the weight is stored as a separate number, and only used when looking at the sub directly - not on aggregate views - this problem can be worked around for the time being. Ideally we'd have a much better ranking algorithm that takes cues from vote weights and the median/mean/average votes posts get in any given subreddit, but that's a topic for another discussion. ;)

This means each user's vote weights are unique to every single sub on the site, based on their subscription date, level of participation, and quality of content submitted to each sub. What you earn in one community will not help you in another community - with the exception of that 'sister subreddit' feature. I think that's important to have to promote groups of subs that all form a single community, such as gaming, television, sports, music, images, etc. The NFL subreddit will want to reward participation in the team subreddits, and vice versa.

A user's effective weight does not need to be calculated in anything near real time, as the vote weight once calculated does not change often. Best suited to some intermittent background processing task.


Now, let's talk about a way to calibrate the vote weights if age and karma aren't doing a proper job. This is a tricky solution because if it's used improperly it could make quite a mess out of the weights. If it's used wisely, however, it'll guarantee the weights go to the people who vote with the sub's best interests in mind.

A 'calibration post' is something that no one interested in the sub's topic matter could ever downvote in good faith. Examples of this would be the genre roundups and yearly best of lists put together in listentothis. Anyone downvoting those is either ignorant of the rules or not interested in the sub's topic matter and shouldn't be subscribed in the first place. Moderators choose what to flag as a calibration post. Regular users remain unaware of them.

The way these work is simple. Reddit tracks these posts in some special way, and keeps track of the users - specifically, did this user upvote more calibration posts than he downvoted, yes or no? Based on the answer to that question, the mods can choose to boost and/or penalize the user's vote weight. The mods might also need a system to 'reset' this record and wipe all subscriber's slates clean.


Once again, the next idea is too large, so this is continued in the next reply...

0
38

[–] Amarok 0 points 38 points (+38|-0) ago 

Solving the TrueTrueReddit Problem (and cutting back on censorship)

Put simply, this is just better content sorting.


Most subreddits have a fundamental problem - they have a very broad, general focus. This leads to endless debates about what is or is not on topic, and different factions fighting over it. This also leads to mod teams that ban entire topics from their subreddits because they don't want to deal with them.

Plenty of subreddits have taken a stab at solving this problem using the flair system. /r/listentothis has a genre sorting bar across the top to filter music by genre. /r/askscience lets you filter the content by area of scientific inquiry. At present this is accomplished by nearly useless flair hacks (and a broke ass search system) that most people can't see thanks to mobile and disabled styles. Those flair hacks do point the way to how this feature ought to work, however.

Imagine a system...

  • of up to 25 moderator-definable content categories
  • which are all listed by name across a new browsing element at the top of each subreddit
  • where each element works like a flair, and can be styled like a flair

That gets us what we have now, but not with a hack. Taking it to the next level...

  • each of these items can be toggled on or off with a visible indicator such as different colored names
  • showing or hiding the content categorized under each type
  • with persistent settings for each user's own preferences
  • that even apply when they are looking at their own front pages

This allows users to be more selective and subscribe to only certain subsets of each subreddit's content. This gives people the power to banish the things they aren't interested in seeing and should lead to a lot less infighting. Moderators can also be more inclusive, designating a category for 'reposts' or 'old news' where they can toss posts that they would otherwise remove. That will cut down dramatically on reddit's perceived censorship problem. This will also cut down on unnecessary subreddit fragmentation. There are good reasons to fork a subreddit but this is not one of them.

Dozens of larger subs have versions of themselves dedicated to questions only, or image macros only. These should be separate tabs of the main subreddit, not forked off into oblivion where no one will see them.

This isn't meant to replace flair. This is meant to supplement it. Then flair can get back to being flair again, rather than a sorting hack.


You made it all the way to the end of that? Good for you. ;)

Full disclosure: I mod listentothis on reddit.

0
1

[–] Strazdas 0 points 1 points (+1|-0) ago 

If i had any upvotes left i would give them all to you. This post is amazing. I had the vague concept of this that i used to call "old forum syndrome" because its a thing that predates reddit but you have fleshed it out amazingly. I have to remember to come here tomorrow and upvote this post because apparently i only get 10 in 24 hours.