We learned a lot from testing the new rating system, but we don't think it's ready for prime time.

Skinner ADMIN

(1000+ posts) Send PM | Profile | Ignore

Thu Sep-25-08 04:23 PM
Original message

We learned a lot from testing the new rating system, but we don't think it's ready for prime time.

Two days ago we unveiled an update to our recommend/rating system. Overall, the response to the change was quite positive -- much more positive than we expected. Most people seemed to be using the system responsibly, and the ratings seemed fair and relatively accurate. I think this test demonstrated that there is value in giving our members a more comprehensive rating function than what we had.

But we do not feel that the new rating system, as we designed it, turned out to be a good fit for DU. So we have decided to go back to the drawing board. We are going to temporarily return to the old "recommend" system that we had before, and stick with it until November or so. We think that unveiling any new upgrades before election day would be too distracting to this community.

So, what good came from this exercise?

We learned that the vast majority of DUers can be trusted to use a comprehensive rating system in a responsible manner. The DU admins felt that most of the ratings were justified and relevant to the content of the discussion threads to which they were assigned.

Even though there was some resistance to the idea of negative ratings, in practice they did add value to the site. I think they acted as a sort-of "pressure valve," so the community could express their disapproval without having to resort to flaming. (Having that option might even create a situation where the moderators do not feel the need to lock as many threads as they do now.) And I think they made it significantly less likely that highly extreme or divisive posts received high ratings. With this in mind, I think it is likely that any future change to the rating system will include the ability to rate threads down somehow, perhaps by letting people "unrecommend" when recommendations have already been given.

And what were some problems?

I think that the system we created was just a little bit too cute. Color-coding, distracting graphics, words assigned to each rating level -- it was unnecessary, and off-putting. Plus, it made the system seem more complicated than it actually was.

It is completely unnecessary (and even pointless) to give people five options to choose from when rating a discussion thread, because everyone just gravitates to the highest and lowest options. Having additional options made rating threads more confusing. I think we really only need two options: one positive and one negative.

The Greatest Page was significantly less useful, or at least it felt less useful. The fact that there was so little stratification among the highly-rated threads made it hard to pick what was supposed to be the "best." Ironically, the "problem" (if you can call it that) was apparently that we actually do have lots and lots of really great threads here on DU. But seeing them all given precisely the same rating of "5" felt deeply unsatisfying to me, and made the greatest page seem unreliable. I felt like I needed to hunt around to try to decipher which item was the "real" greatest thread. Also, removing the column in which the greatest threads were sorted by time was a mistake that made the page less useful.

So, we're taking down the new, improved rating system, and going back to the old tried-and-true "recommend" function. The old recommend function is not perfect by any means. There are many things about it that I do not like. But it is what we know, and we have already become tolerant of it's shortcomings. It should be sufficient to get us through election day, when Barack Obama is elected the 44th president of the United States.

Thank you for your patience and understanding as we try new things and work to improve this great community.

Skinner
DU Administrator