There is something seriously ridiculous about AGS reviews

[jahnocli]

Your subjective view of the worth of your games clashes with other people's subjective views. No-one can win an argument like this.

[Khris]

I just looked at the panel ratings of my own games and they are all spot on

@lapsking: Chill, the main point of the ratings is to convey whether a game is worth the effort of downloading and playing it or not. Both your games have been deemed worthy. In no way is the rating telling you that all the effort you put in your second game was a waste of time, that's only happening in your head.

Anybody who plays both games will clearly see how much more work and polish went into the 2nd one, so why do you care that much about the cup rating?

[Babar]

I'm not exactly sure what specifically you are complaining about here?

There is no such thing as an "objective review". Even if all the reviewers were following strict guidelines about how they score their reviews, it will always be coloured by their own subjective opinions.

The review comment is where a comment is given to explain the rating, and to provide more context. The panel rating for your game The Order was actually positive. The ONLY somewhat negative comment (if you take it that way?) is that your art wasn't of the highest quality (but it was still complimented about it being done by hand). Would you have preferred that the Panel Rating for "The Will" actually wrote "This game has really bad art"? Unless the game is obviously bad in some way, Panel Rating comments stay away from negative comments like that.

I haven't played either game, but it shouldn't bother you that even with your less skilled art from The Will, it ranks in the same ratings number as your improved skill art from The Order. There are lots of games like that, and graphics is just 1 part of the rating.
Here are some other games with worse and better graphics than your game, also rated 3:
https://www.adventuregamestudio.co.uk/site/games/game/2314-tales-from-the-outer-zone-the-goat-crone/
https://www.adventuregamestudio.co.uk/site/games/game/2276-space-tunneler-deluxe/
https://www.adventuregamestudio.co.uk/site/games/game/1223-star-trek-newton-part-one-anomaly/
https://www.adventuregamestudio.co.uk/site/games/game/2604-saturday-night-is-the-loneliest-night-of-the-week/
https://www.adventuregamestudio.co.uk/site/games/game/1010-nes-quest/
https://www.adventuregamestudio.co.uk/site/games/game/2190-10-ways-from-sunday/

[heltenjon]

lapsking, please don't alienate everyone.

@ThreeOhFour and @WHAM has this slightly wrong, I guess. The bell curve will apply to user reviews (the orange cups). When or if enough players have rated a game, this will give an average like what you describe.

The panel scale, as explained on the games page and in the sticky post from Andail (which may be outdated), operates slightly differently. 2 cups is average, and most games get that. 1 cup games has serious shortcomings of some sort, while 3 cups are good or recommended games. The exact text on the games page:

The AGS Panel is a team of AGS forum veterans who play all of the games here, and give them ratings from one to five cups. The rating system is as follows:
1 Cup    Not serious entertainment
2 Cups    A reasonable game, worth a try
3 Cups    A good game, worth playing
4 Cups    A great game, well worth your time
5 Cups    An outstanding, must-play game!
Feel free to disagree with the panel's rating, and please don't be offended if you think your game has been underrated!

For comparison, the player ratings are worded like this:

1 Cup A bad game, I wouldn't recommend it
2 cups Play it if you're bored, but not much fun to be had here
3 cups It was ok, play it if you've got some spare time
4 cups Recommended, definitely put it on your to-play list
5 cups Amazing! You have GOT to play this game RIGHT NOW!

The way I interpret this, 3 blue cups equals about 4 orange cups. The player ratings tend to be higher than the panel ratings for many or most games. (Note that this is just my impression, I don't have the statistics.)

Now, occasionally, the panel will f up, misunderstand a game or give the "wrong" rating for some reason. This entire forum is filled with people doing this on their free time without getting paid for it. So the panel, the moderators, game devs, engine devs or coding helpers are all in the same boat, so to speak. In this case, I'll agree partly with lapsking and say that his first game probably was rated a bit generously by the panel due to the art and sound shortcomings he's referring to. I remember liking the puzzles a lot, though, so crap it was not.

As for changing the rating system to a different scale (0-10), that would probably mean losing all the player ratings through many years, in addition to having to rerate several thousand games. I don't think that's going to happen.

I can only speak for myself, but when I give a player review, I'll definitely take into account stuff like game length or genre. I'll compare a short game to other short games, I guess, like I would not compare a poem or a short story to a novel - they're different experiences. Likewise a comedy game ought to be funny and a horror game scary. If it's the other way around, (like unintentionally funny) I won't score it that high. If I paid money for the game, probably something like value for money will come into it, too.

[lapsking]

I wasn't trying to alienate everyone. And I'm glad I brought it up. It's just the black duck among white geese story. And I can't stand nonesense as much as you can't.

[Snarky]

A guy goes to a karaoke night in a bar and sings a song. People clap and there are some cheers. Encouraged, he starts practicing, and six months later he comes back with a band as the evening's musical performer. He gets about the same amount of applause as when he sang karaoke.

Obviously he is furious. 

[lapsking]

A guy goes to a karaoke night in a bar and sings a song. People clap and there are some cheers. Encouraged, he starts practicing, and six months later he comes back with a band as the evening's musical performer. He gets about the same amount of applause as when he sang karaoke.

Obviously he is furious. 

Hahaha, ha, you are so funny Snarky. Maybe try to make a dark humor game. Get busy with your modules. 

[Danvzare]

Honestly, I don't see why you're complaining. It's just a subjective review that's highly dependent on the reviewer.

I mean look at my games.
Rowen Goes To Work (2009): "The game is short, uses borrowed graphics and music, and most unfortunately, the puzzles tend to be completely illogical, like using soap in a microwave to make a key. "
2 cups
Crepe Fields: A Scare Among Crows (2021): "While the graphical presentation and voice acting is average, the mystery writing and investigative puzzles are strong points in this adventure."
2 cups

The first game is terrible (I mean it was my first) and the blurb showcases that. The second game is quite good and the blurb showcases that as well. But they have the same rating. Why? Who cares! Someone played my games and cared enough to review them, that's all that matters. The fact that they both got 2 cups is meaningless to me, since I didn't make the game to appease them specifically. Especially since something like a rating is so subjective that even two people who share the exact same opinion on a game can give it different ratings wholly dependent on what they consider each rating to mean.
I mean just look at normal game reviews, where anything under a 7.0 is typically considered to be equal to a 0.0. 

Your self-worth should not determined by the opinions of others. 

[lapsking]

Honestly, I don't see why you're complaining. It's just a subjective review that's highly dependent on the reviewer.

I mean look at my games.
Rowen Goes To Work (2009): "The game is short, uses borrowed graphics and music, and most unfortunately, the puzzles tend to be completely illogical, like using soap in a microwave to make a key. "
2 cups
Crepe Fields: A Scare Among Crows (2021): "While the graphical presentation and voice acting is average, the mystery writing and investigative puzzles are strong points in this adventure."
2 cups

The first game is terrible (I mean it was my first) and the blurb showcases that. The second game is quite good and the blurb showcases that as well. But they have the same rating. Why? Who cares! Someone played my games and cared enough to review them, that's all that matters. The fact that they both got 2 cups is meaningless to me, since I didn't make the game to appease them specifically. Especially since something like a rating is so subjective that even two people who share the exact same opinion on a game can give it different ratings wholly dependent on what they consider each rating to mean.
I mean just look at normal game reviews, where anything under a 7.0 is typically considered to be equal to a 0.0. 

Your self-worth should not determined by the opinions of others. 

Exactly, at last someone is saying what I was saying all the time. IT'S MEANINGLESS. Thank you my friend.

[lapsking]

I just looked at the panel ratings of my own games and they are all spot on

@lapsking: Chill, the main point of the ratings is to convey whether a game is worth the effort of downloading and playing it or not. Both your games have been deemed worthy. In no way is the rating telling you that all the effort you put in your second game was a waste of time, that's only happening in your head.

Anybody who plays both games will clearly see how much more work and polish went into the 2nd one, so why do you care that much about the cup rating?

Maybe because I was taking AGS too serious my friend? Maybe I was shocked that your coding does worth as my crappy if this then then.

[LimpingFish]

I just very confused by your anger. This is not an exact science. I've explained how I personally rate games, not because I feel it needs defending, but because you seem determined to find fault with the process.

The Order was rated be me. I liked it. I found it strong in some areas and lacking in others. The art style (your painting) initially struck me as somewhat rudimentary, but it grew on me the further I played, despite it's simplicity. I admire the fact that you painted the entire game, but how you achieved said art, or how long it may have took you, bore no impact on my rating, nor should it. And while rudimentary, it was consistent, which is a plus.

You seem to take umbrage with me expressing my opinion, when it is simply just my opinion. And I don't see how I could have addressed your art without forming an opinion. The panel comment, which you angrily posted about, reflects this:

"And while the art isn't strictly of the highest quality, the artist must be commended for their dedication to painting it all by hand."

Like I said in my earlier post, if you decide your next game has better art, does that mean I was actually right in my opinion?

I didn't rate your first game, but the person who did obviously found something worthwhile contained in it, and did so without having your second game to compare it against. You seem determined to have your art judged according to how much value you, the artist, confer on it, and in doing so, you may be setting yourself up for constant disappointment.

And even if I had rated your first game, I can assure you the quality of that first game would not have effected my rating of the second. I may have noted in the panel comment that your second game was an improvement on your first, but I wouldn't have increased my rating, or retroactively reduced the rating of your first game.

[lapsking]

So now we are getting somewhere. Someone rated my first game, and another person rated my second game. Do you see any science in this kind of rating? Does this kind of rating system deserve 3 pages defending? Is this comparison among AGS database by any means slightly accurate? Then there is something seriously ridiculous about AGS rating system. Rating can be like IMDB just by audience. But if you want to be rotten tomatoe and give AGS panel review or rating, this system is totally absurd. If you don't have the resources, just skip AGS panel rating and review, it can be easily misrepresenting and false. That's where my anger comes from, otherwise Snarky's joke and others silly excuses just makes AGS look more like a ridiculous circus. What if you like horror and the previous one likes romance. So actually from what I gather, the previous review was one person's review, and this review is your review. In my wildest imagination still I can't call this AGS panel review. I'm really appreciate that you are taking responsibility for your review, but maybe from the beginning you needed to write your name instead of AGS panel. And he/she could write his/her name. It gives a feeling that there is a group of high professional jury which is rating and reviewing the games. Or maybe you could also use audience rating system to rate my game.

[lapsking]

I have another question. Are you guys as a "community" having "picnic" together? Or are you into something more serious? Because I have no intention to ruin your picnic which I do all the time and which is why people avoid inviting me to their picnics. I don't mean to be "traffic". 

[Crimson Wizard]

I have another question. Are you guys as a "community" having "picnic" together? Or are you into something more serious? Because I have no intention to ruin your picnic which I do all the time and which is why people avoid inviting me to their picnics. I don't mean to be "traffic".

I did not want to reply to this thread, as I've never had anything to do with the website, ratings etc, but the further this goes the more disgusting this becomes. This thread is no longer the issue of the ratings, it's the issue of your attitude. It is perfectly clear at this point that you have zero respect for other people here, and you keep emphasizing this with nearly every new comment, towards ones who did nothing wrong to you.

This "community" is simply people who enjoy making games with certain game engine and playing them, nothing else. Nobody here ever tried to claim that it's anything bigger.

Just above you said that, I quote, "silly excuses just makes AGS look more like a ridiculous circus".

I think that the only thing ridiculous here is you. You are an annoying, tedious, overly pretentious <removed by moderator>, who got triggered by a hobbyist community rating.

[Snarky]

from the beginning you needed to write your name instead of AGS panel. And he/she could write his/her name.

The argument for keeping panel ratings anonymous is that it allows the reviewer to be critical without risking the ire of some offended game maker taking it personally.

IIRC, I argued against that position at the time, but your reaction kind of proves the point.

[lapsking]

from the beginning you needed to write your name instead of AGS panel. And he/she could write his/her name.

The argument for keeping panel ratings anonymous is that it allows the reviewer to be critical without risking the ire of some offended game maker taking it personally. So you wanted to keep the reviewer anonymous and instead give a false value to the reviews?

IIRC, I argued against that position at the time, but your reaction kind of proves the point.

Do you know what does AGS "PANEL" mean? Fortunately, my mother language is not English and I had to look it up in dictionary. Maybe you should do the same thing.

[lapsking]

from the beginning you needed to write your name instead of AGS panel. And he/she could write his/her name.

The argument for keeping panel ratings anonymous is that it allows the reviewer to be critical without risking the ire of some offended game maker taking it personally. So you wanted to keep the reviewer anonymous and instead give a false value to the reviews?

IIRC, I argued against that position at the time, but your reaction kind of proves the point.

Do you know what does AGS "PANEL" mean? Fortunately, my mother language is not English and I had to look it up in dictionary. Maybe you should do the same thing. Maybe "The Order" got to 2 cups if it was really reviewed by so-called non-existence AGS panel. And The Will got 4 cups. Then I would be happy.

[Babar]

I'm even more confused with your comments now. I am guessing (hoping?) that the language barrier is responsible for most of the issues, because otherwise it makes no sense.

The AGS community doesn't know each other. We don't go on picnics together without inviting you and make fun of you while munching egg sandwiches. We are all here because we enjoy making and playing adventure games, the same as you, I'm assuming.

None of the panel members are paid for their work. None of the developers who work on making AGS are paid for their work. There is a donation button, and that mostly goes to server costs. Every person who rated your game did it on a voluntary basis, on their own time, with their own effort. If you want, you can also PM the mods and request to join the panel to rate games. How exactly do you think a rating system like that of rottentomatoes (which you mentioned) works? The only difference here is that those critics get paid for their work.

Do you think that AGS members or panel members are so dumb that situations like "Oh, I like romance games, and this is a horror game, so I will rate it low" arise? How exactly would you envision a "better" "perfect" "proper" AGS panel rating system? Can you outline what specifically are you so annoyed with in the current system (besides the fact that they gave 2 ratings that you personally don't agree with)?

[lapsking]

I'm even more confused with your comments now. I am guessing (hoping?) that the language barrier is responsible for most of the issues, because otherwise it makes no sense.

The AGS community doesn't know each other. We don't go on picnics together without inviting you and make fun of you while munching egg sandwiches. We are all here because we enjoy making and playing adventure games, the same as you, I'm assuming.

None of the panel members are paid for their work. None of the developers who work on making AGS are paid for their work. There is a donation button, and that mostly goes to server costs. Every person who rated your game did it on a voluntary basis, on their own time, with their own effort. If you want, you can also PM the mods and request to join the panel to rate games. How exactly do you think a rating system like that of rottentomatoes (which you mentioned) works? The only difference here is that those critics get paid for their work.

Do you think that AGS members or panel members are so dumb that situations like "Oh, I like romance games, and this is a horror game, so I will rate it low" arise? How exactly would you envision a "better" "perfect" "proper" AGS panel rating system? Can you outline what specifically are you so annoyed with in the current system (besides the fact that they gave 2 ratings that you personally don't agree with)?

Yes, I think English language is a barrier. Cause panel in my dictionary means a group of people1.
a flat or curved component, typically rectangular, that forms or is set into the surface of a door, wall, or ceiling.
"a layer of insulating material should be placed between the panels and the wall"

I'm even more confused with your comments now. I am guessing (hoping?) that the language barrier is responsible for most of the issues, because otherwise it makes no sense.

The AGS community doesn't know each other. We don't go on picnics together without inviting you and make fun of you while munching egg sandwiches. We are all here because we enjoy making and playing adventure games, the same as you, I'm assuming.

None of the panel members are paid for their work. None of the developers who work on making AGS are paid for their work. There is a donation button, and that mostly goes to server costs. Every person who rated your game did it on a voluntary basis, on their own time, with their own effort. If you want, you can also PM the mods and request to join the panel to rate games. How exactly do you think a rating system like that of rottentomatoes (which you mentioned) works? The only difference here is that those critics get paid for their work.

Do you think that AGS members or panel members are so dumb that situations like "Oh, I like romance games, and this is a horror game, so I will rate it low" arise? How exactly would you envision a "better" "perfect" "proper" AGS panel rating system? Can you outline what specifically are you so annoyed with in the current system (besides the fact that they gave 2 ratings that you personally don't agree with)?

Yes, I think English language is a barrier. In my Google dictionary it means:

1. a flat board on which instruments or controls are fixed.

2. a small group of people brought together to investigate or decide on a particular matter.

I don't know but if a group people didn't play, rate and review my both games, then I think AGS panel must be a flat board or something.

[Snarky]

Do you know what does AGS "PANEL" mean? Fortunately, my mother language is not English and I had to look it up in dictionary. Maybe you should do the same thing.

Yes, I do know. There is a panel, a group of people, who have been selected as reviewers. One of those reviewers reviews each game. The fact that each game is reviewed by only one panel member has been discussed extensively in the past.

You didn't know, and that is perhaps a signal that what it means could be communicated more clearly, but it's a perfectly valid term for the system in place.