Rating system

[Dualnames]

People should be able to disable AGS rating comments and panels or contact the guy who rated them and ask the reason. I find that a 3 cup rating on a game that is praised by 90% of the people to be 10/10, not to receive a 4 cup (defines the game as a must-play), to be unjust. As unjust is that H2G2 towel day got rated 3 cups though it can be enjoyable and breakdown by one dollar was rated with two stars because it had 1 puzzle that was a little hard. Difficulty and length has nothing to do with a game's quality. EVER. Shifter's Box is like a two hour game MAX, though it's a really enjoyable experience. While other games last more, and offer nothing. I've never really bothered about the rating system, but certainly seems unfair for this game.

[Calin Leafshade]

if you can disable it whats the point? everyone who got less than 3 cups would disable it.

[Andail]

Dualnames, I wasn't able to understand half of your post.

As unjust is that H2G2 towel day got rated 3 cups though it can be enjoyable and breakdown by one dollar was rated with two stars because it had 1 puzzle that was a little hard

Are you saying it's "unjust" that your game got 3 cups...because it's enjoyable? Don't get it. Is it worth more? Or less?
And is Breakdown's rating unfair because of...what? The panel left a comment to explain the rating. Apparently the panel member found that "Shows a lot of promise but the gameplay consists of more than a few tedious pixel hunts or illogical/strange solutions."

Difficulty and length has nothing to do with a game's quality. EVER.

Don't you agree that a game shouldn't be so hard that the player can't progress at a reasonable pace, nor easy enough to let him win without trying or thinking?

Also don't you agree that a game should have at least a modicum of content before it ends, to let the player get into the plot and the characters?

I think everyone needs to remember, before you jump on the complaint bandwagon, that the descripition for 2 cups is that the game is reasonable and worth a try.

[Dualnames]

No wonder Andail.

I just find that H2G2 Towel Day got a three cup rating, and I considered it short of fair. I however seem to find unfair to have a game (MCC), rated only 3 cups. Perhaps you are right though. I find reason in your points, and to be fair you are right.

I never did find a rating to be unjust concerning to the comment it had. However it's like focusing on the cons of the game. It's either wrong to focus on the pros and the cons of the game and rate it based on that.

As for difficulty, indeed the puzzles should be REASONABLE.People nowadays forget that. I detest pixel hunt except if it's done right.

[IndieBoy]

Gah! I never get involved with these random forum rants:

I always thought the ratings were down to organising the database and not a specific, 100% accurate and official rating made by "AGS elite". Such an organisation is only to really benefit non-ags community members and non-active community members, which will tend to only play the 4/5 cup rated games (generally). Any other game with whatever rating is still played by the community and will get the same exposure as any rated game on the database (excluding the honour and promotion of pick of the month and the awards). I mean people aren't going to be put off by a rating, I wouldn't be and this is also mentioned by a few members in this thread. Evidence is mags games, hourgames, orow games, currently unrated games, currently popular game threads etc.

Now disabling the rating after over a thousand games have been individually played and rated over the last few years, with the same system is silly and is rather insulting to the panel who have taken the time to do a service to the community which is un-paid.

Although saying this nothing is perfect, and it's going to be near impossible to make a 100% consistent system, due to the nature of the community and the medium that is games which is all rather subjective. I am totally honoured, with my demo (which was originally a short game rated 1 cup) and the 3 cup rated mags game I was involved with, to be in the database and was individually rated. However I am not taking it as a level of stature which I come think people come to desire, i.e I'm not screaming my ratings for my games in the street.

Anyway the database is controlled by a panel, let them go do what they want with their database. The awards, the threads in forums, personal blogs and countless other review type sites is down to everyone else. And with this whole thing been started by Calin's game I'm sure, by the recent publicity he will be a strong contender in these up and coming awards, which the community decide.

Games are mean't to be fun, why can't we be all light hearted.. and fun?

[Dualnames]

I've never said anything about my games because they deserved every rating they've got. And I find it more or less fair, so it's up to Calin, to say if he feels that the rating he got is unfair. I'm lighthearted alright. Hell, for all I care, I'm mostly proud of them, even those that got a 1 cup rating.

[Calin Leafshade]

I didnt feel the 2 cup rating was that unfair.. it seemed reasonable enough after the initial "thats bullshit" moment.

The 3 cup rating seems perfectly justified to me and I don't feel the game has merit to be a 4 cup game by any criteria.

[m0ds]

People seem to be forgetting that 2 cups is still a good rating. I don't know. My game is 2 cups and I'm thrilled with that.

As for why the panel should remain anynomous etc, simply cos none of the panel members should be asked to justify what they're doing nor should their personal lives be interupted because of it. A system is in place, it has guidelines, that is that! You really don't have to put your games on the db if you think your cup rating is going to affect downloads or attention.

I fail to see the point. I'm not on the panel but can't really see what the problem could be. What's to hide or protect?

Nothing to hide or protect, it just saves time. Perhaps there does need to be one point of contact, a bit like an AGS OFCOM so that where people feel the rating isnt right, be it about cups or where say violence hasn't been correctly rated - to vent it directly to the panel - but that would be a system like an email form, because again I don't think anyone really wants to spend the extra time discussing all the rating and panel stuff.

The panel try to be un-biassed and just follow a standard system, again maybe there needs to be an explanation of it on the page somewhere - but we can't expect everyone to play every game ever made in order to get a fair comment, and so in a lot of cases the reviewer will have to make some judgements by themselves. That person doesn't always feel its fair but I know everyone that does the panel is just following a system well before any personal preferences come in - but alas, you can't escape the fact those preferences will in cases effect the score because 2 panel members doesn't really justify 2000 players. But that's the way it is. It's not hiding anything.

The only thing it protects is privacy. I don't want you bugging me about my affiliation with the panel, and I'm sure many of the others that help on it don't want to be pestered about it either. It's a bit of fun and we all knew from the offset that some people would not like the ratings in some cases - and lo & behold it does happen occasionally, people have removed all their games because of one bad rating, pulled their downloads and undoubtedly done other things I don't know about. But the serious game developers just keep going, or just publish games to their own websites.

[auriond]

The reason why people forget a two cup rating is still good is that it's only one step up from "Not serious entertainment". Also that 2/5 = 40%. "Sorry kid... you missed the halfway mark by that much. You nearly made it though. For serious. Welp, better luck next time."

Not much else more to say at this point except to wait for official word on whether those mysterious criteria will be made public.

[LimpingFish]

Not much else more to say at this point except to wait for official word on whether those mysterious criteria will be made public.

There is no mystery.

And some games do indeed miss the 3 cup mark by that much. In some cases it goes to a discussion. Sometimes a majority supports the 3 cup, sometimes the 2 cup.

We always knew the 2 cup rating was going to be the biggest opinion divider, but we accept that.

It may indeed only be one step up from a 1 cup, but it's also only one step down from a 3 cup. Cup half empty, cup half full.

I think I can safely say that no explanation on the subject of what goes into rating a game will be forthcoming, since it would benefit no one except those who would wish to pick it apart. Although I can make up a secret formula, if people need one:

Effort + Talent * Excellence = 5 cups! \o/

Seriously, if you enter into making a game trying to second guess someone else's opinion, you not going to enjoy the experience.

[Ryan Timothy B]

Which is one reason why I believe:
1/5 cups should be for joke games, poorly made games, or games that aren't really even worthwhile or broken.  
2/5 cups should be for somewhat entertaining games with little content (in some areas perhaps lacking dialog, interactions, animations, a story or other important elements)
and obviously 3/5 being for the average game

These are free games, so obviously we shouldn't be too strict on the rating but also not too lenient either since honest ratings will help promote the games that deserve the higher rating, and in the end they'll promote AGS; giving us more games!!

[auriond]

I must have missed the post where they were made public then... *goes back to look*

Half-empty, half-full indeed. Well, we can't all be optimists.

[Baron]

I agree with Ryan Timothy, that it is intuitive to think of 3/5 as average.  The bell-curve of games in the database is heavily skewed in that there are very few 4/5 and 5/5 games.  Perhaps those categories should have been merged into a "really good" 4/4 or 5/5 category....  But it's all pointless to debate it now, since you'll never muster the enthusiasm to go back and re-rate the 900+ games already in the database.  For consistency's sake we are more or less stuck with the status quo, so us game makers are just going to have to roll with the punches.  If a game has been perceived to be slighted, recent evidence suggests that raising a ruckus will correct the problem -that's good enough for me.

I think a section where you could provide links to reviews in the games database is a wonderful idea, but it will have to be author driven.  Any serious potential player would rely more on in-depth reviews than some ill-explained rating system when deciding whether or not to download, and the sorry fact of the matter is that the non-serious player is just going to judge the game on its screen-shot anyway. 

[Igor Hardy]

These are free games, so obviously we shouldn't be too strict on the rating but also not too lenient either since honest ratings will help promote the games that deserve the higher rating, and in the end they'll promote AGS; giving us more games!!

By saying that, you're demeaning free games. Quite a few of them are much better than the majority of commercial ones. I say, be strict with the ratings, and make the best games stand out as much as possible.

Btw When I'm browsing the database I read the number of cups like this:

1/5Cup - probably a really badly put together game
2/5Cup - there are some serious problems with this game
3/5Cup - this game could be anything in terms of gameplay quality, but at least you can be sure it works
4/5Cup - a stand-out game
5/5Cup - a stand-out game Type 2
Pick of The Month - a stand-out game Type SSH

Usually I have a hard time taking notice of the User Ratings. Too many percents, so I forget them all in 2 seconds. I take note of how many people voted and commented though.

And in the end I don't pay much attention to the ratings, just download the games which screenshots appeal to me (unless the game info is off-putting). And then usually I don't have time to play them all, so I pick out some of them randomly from my AGS games folder.

[Ryan Timothy B]

By saying that, you're demeaning free games. Quite a few of them are much better than the majority of commercial ones.

By commercial games, you mean AGS ones right?  Because I haven't played an AGS game yet that is better than a big title Adventure game (Sierra, Lucasarts, etc).

And yes, I am demeaning free games slightly, but you can't honestly tell me that a 5 cup AGS game means: It's as good as Freddy Pharkas, Monkey Island, DOTT, etc.  It just means it's a damn good 'AGS' game, or at least it's how I view it.  Although the Apprentice game comes pretty close, and I haven't played the KQII+ VGA yet.

[Calin Leafshade]

In terms of production values AGS games cant compete since commercial games are developed on a full time basis.

However in terms of enjoyment and a compelling tale I think I prefer trilby's notes to DOTT.

To say that a 'big budget' game cant compete in terms of the narrative is baseless as far as i can see.

[Helme]

Quite a few of them are much better than the majority of commercial ones.

Seconded! AGS-games can't compete in terms in graphics and voice acting, but in terms of puzzles and story-telling.

For example sometime ago I played forced myself to spend time with Runaway 2 after I payed money for it. I can't remember much AGS-games with such crappy puzzle design.

[kaputtnik]

Useful intervention!

Hasn't anybody ever considered that the fine line between crap and genius can't be expressed within a system of five (5) different grades? It just can't!

You read a review in some games magazine that says:

"72%. Great motocycle physics, but holes in the track and lacking spraypaint mode subtract somewhat from the overall enjoyment."

Then you read the next one.

"81%. Motocycle tuning mode works really well, racing is lots of fun for some hours. Only the sometimes uninspired career mode dialogs prevent a higher rating."

Which one would you choose? Do 9 points on a 100 point scale make a difference? I do not have any suggestion to solve this problem, of course, but I think any rating system with less than ten different grades is necessarily inaccurate. While 2 cups are certainly intended to express encouragement, they do put most players off.

[Igor Hardy]

While 2 cups are certainly intended to express encouragement, they do put most players off.

I don't think it's the 2 cups themselves that put most players off. 2 cup games can be off putting only in the context that so many other games you can play that have received 3 cups or more. It's usually all about having to make choices because of a limited time.

[Babar]

It is my understanding that the ratings system is first and foremost an attempt to organise the games database, more than to make judgement calls on individual games. It is true that a 1-5 rating cannot completely encompass everything a game has to offer, but realistically, neither does a 1-10 rating. You might as well go to the absurd extreme: from a cursory glance, it seems there are about 1100 entries in the games database. Should there be 1100 gradations in the games ratings? Then you'd be getting even more arguments about how "This game is not better than that game!" and it'd all become a competition.