This post is cross posted from GameMarx.com
Background: In August of 2009 Microsoft updated the Xbox Dashboard to allow user ratings on games. Along with the ratings came a set of "Top Rated" lists for the various categories Microsoft places games into (Arcade, On Demand, Xbox Live Indie Games, etc). Gamers could rate games in their library and in the Marketplace, which is somewhat controversial, as there is no requirement to download a game let alone play it before giving the game a rating.
In November of 2010 Microsoft updated the Xbox Dashboard, but this time they left out rating a game from the game library. Now to rate a game, of any type, you must find the game in the marketplace. This means you have to find the game again to rate it if you are the type of person who download and plays a game before forming an opinion. I would guess that only someone who hates or loves a game, i.e. only the 1 star and 5 star voters will feel the desire to put forth the extra effort. The average gamer will just not bother.
To see the change between dashboards, here are before and after screenshots of an Indie Game in the game library:
Keeping up with games at GameMarx I’d started to notice many games have a lot less votes than before. I will see a game come across with 4 or more stars but only has 20 votes behind that rating. This is good if you are a developer with a large social network following – get people to vote and it won’t take many to grab a spot on the Top Rated list. While I would agree that a developer with a large social network has probably earned it by creating quality titles, this isn’t the intent of the rating system. Still, a feeling is just a feeling and may not be reality, so I made a chart:
I took the games and grouped them by month of release, then figured out the average number of votes games got that month. It’s pretty cool to see how the number of games released in a month is trending up. It’s not cool to see that blue line trending down to nothing. When looking at the vote average it’s important to remember that the date is the the game’s release, not the date the vote was cast. It would be natural for new games to have less as they will gain votes over time. Because of this I don’t think there is enough data to say with certainty the dashboard update impacted votes. Another explanation is that the number of games is increasing, but there are no new gamers finding the Indie Games section. That might be true, as the issues I’ve written on in the past still exist, and Microsoft after a brief breach of protocol in November has gone back to the stone cold silence policy for community communication.
Why is all this important? Imagine the Apple App Store having no way to filter on user ratings. It would be chaos, a mess to find the good apps. The same hold for Indie Games. The Marketplace needs to help gamers quickly locate the good games among the 1,600+ titles available today. If the system isn’t getting good rating data, it will be hard to surface great games. There is also a Top Downloads list, that is believed to be a bestselling list, but I have my doubts. Unless last week the number two selling game was an incredibly awful Beer Pong game. (Which has a 3.5 rating?)
Then again maybe things haven’t changed and 1Up’s Scott Sharkey is still right. You need to be bored and drunk to find Indie Games and thus drinking games sell well.
(Another interesting tidbit, I MAED A GAM3 W1TH Z0MB1ES!!!1 – the best selling and highest rated game on Indie Games – came out just a few weeks after user ratings were added to Xbox. Sounds like something I would read in a Malcolm Gladwell book)