There is something seriously ridiculous about AGS reviews

[lapsking]

But rating my first game the same as the second game just makes me wonder if they have any clue how they are reviewing the games. If they gave the first game 1 cup and the second game 3 cups it would make sense. But this is ridiculous.

Why? Do all artists inherently make better art as time progresses? And by "better", do we mean more polished? Is polished art better than unpolished art?

Maybe I don't have a clue what I'm making, and the crap that I made to learn how AGS works is as valuable as something I put more than two years of my life in it.

If your first game was crap, why did you release it? Did you know it was crap when you released it, or are you saying that you now consider it crap? Did you want people to play your crap, and call it crap, just so you could release a second game to earn kudos for how much better the second game is compared to the first? Will you retroactively announce this second game is crap as soon as you release a third one? Do I you see why some are saying this is a bad look for you, or do I have to elaborate on this dumb excuse for a gag any further?

I do think there is a bit of a problem with the Panel ratings

Fair enough, but some of your points raise their own problems. Such as, what is the acceptable granulation level for ratings? Half cups? Quarter cups? Ratings comprised of .1 increments? Just what would be the differences between a 3.4 and a 3.6 cup game? I'll go further, and ask what the difference between a 3 and 3.5 cup game is? To me, all that half tells me is that the game is good, but wasn't good enough to be a four? Maybe that's just me, though.

As to MAGS or commercial games, are we talking about handicaps or demerits? Should we be harsher on commercial games, or more forgiving of MAGS games?

Personally, if a game is worth playing, I don't feel the need to examine how it came into being. Factors such as development time, or the number of people involved shouldn't, to me anyway, bear any influence on the most important factor: enjoyment.

As to panel transparency, the panel was always intended to be anonymous, but I've long since waved my own anonymity. And while we do have guidelines, and we do occasionally discuss ratings as a whole, we also trust the abilities of each other to be fair and knowledgeable on the subject.

I think people should remember that 3 cups = Good. Numbers before 3 are less good. Numbers after 3 are...more good.

If your game gets 3 cups, you made a good game.

I deleted the comment from here cause it wasn't a reply to the "quote". Sorry!

[lapsking]

I try to avoid your off-topic traps, cause I'm already convinced The Will and The Order both deserve 3 solid cups. But I don't know why you guys don't understand my point which is very simple. I never argued The Order deserves more than 3 cups. But if a website wants to rate video games, I would like to to know why The Will also deserves 3 cups. Is it because it's shorter in length? Or is it because I photoshopped some online pictures for visual art? Is it because it doesn't have music? Is it because it doesn't have much story (kind of shooter puzzles)? Is the animation a bonus (which is only a drop of water falling from the pipe?). Do you guys prefer 3rd person? Or because it was cheesy? Is it because it has the most simple coding? Or is low-resouloution a privilege on AGS (Pixel Art)? When I say it looks ridiculous to me, I'm not joking. Anyway, maybe it's my fault that I'm not much aware how reviews and ratings work. But it's very funny. I was proud The Will had 3 cups, but now I have this feeling it was a joke, it's all subjective and meaningless when two incomparable video games both get 3 solid cups. Which is a relief, cause when I realized how funny galleries judge art, I focused even more freely on my art. I had a very good laugh, hope you all had one too.

Edit: Oh, to be honest, now I remember another animation, a box moving from x110 to x80. Did that make Creamy happy? 

[ThreeOhFour]

I think one issue here is that you are comparing one game of yours to another game of yours. The ratings panel is comparing a much wider range of games. If you believe your first game to be worthy of a 1 cup rating, have you played other games rated 1 cup, and seen what the panel is comparing it to? Have you played some 2 cup games?

Imagine you have an author that you like. You go back and read their very first book and you think "Okay, that's not as good as their fifth book, which is one of my favourites, I can see the improvement". But you will probably like it much better than some other author's best book. The panel does not exist to compare your first game to your second game. It exists to compare the full range of games people submit to the database.

[lapsking]

I'm just comparing my own game with my own test game and these are the answers I get. God forbidden, imagine what would happen if I compared my game with someone else's game. Anyway, my first game is the most safest to compare and also from AGS database. And probably the most obvious.

[ThreeOhFour]

God forbidden, imagine what would happen if I compared my game with someone else's game.

This is the entire point of the panel ratings, except they are comparing your game with everybody else's game*, not just one other person's game. And this is how all game (and film, etc) review scores work.


*made in a similar style. They probably aren't comparing your game to games that are completely dissimilar, such as a racing or strategy game.

[lapsking]

Where did you read that? You mean Hitchcock's crap gets the same rating as his better movie? Not that I compare myself with Hitchcock, but I wasn't aware rating system is this stupid. I'm just afraid if I make a slightly better game it might get 2 cups, a tad confused.

[ThreeOhFour]

I see this in your first post:

I'm not comparing them to another developer's games which might make it subjective.

Understand this: Reviews are subjective. They are an opinion.

[lapsking]

So your last try didn't work, (comparing my own game with my own game), now you are changing the tracks? That's how I like it. I've got a handful of convincing answers, from all different aspects. It will take me a month to comprehend them all.

[ThreeOhFour]

What don't you understand though? Here's how reviews work:

1. People play your game.
2. They form an opinion of that experience.
3. They compare that opinion to their opinion of every other similar experience in the database.
4. They try to assign that comparison to a very limited scale.

Where are you confused about this process?

[lapsking]

How did The Will comparison with other games on AGS database ended up with 3 cups and The Order comparison with other games on AGS database also ended up with 3 solid cups. I mean the database is almost the same, so does it mean my games are also the same. I would like to know how AGS panel came to this conclusion that both games are 3 cups. However I try to compare I don't come to this conclusion. Please enlighten me only if you have played both games.

[ThreeOhFour]

[lapsking]

I'm not that retarded, I know 3 solid cups is average. This comprehesive statics were the last piece of puzzle that I was looking for to completely understand why my both games have the same ratings. I think everybody is convinced now and we all can have go to sleep.

[WHAM]

Since reviews are always subjective, when using a 5-star (5-cup) system of grading it seems pretty human and natural for a lot of games to end up with a score of 3. I feel this mostly comes down to reviewers, whether it be the AGS panel or individuals, being polite. A 3 is a solid effort that doesn't stand out. A 2 would be a weak performance but the reviewer felt it had some merit, while a 4 is exceptionally good. A score of 1 is usually reserved for either personal issues (this game has X and I don't like X so I'll give it a 1 out of principle) or in the case of the panel reviews a genuine belief that the game lacks any considerable effort or skill in its makeup. Meanwhile a score of 5 is practically a superlative, reserved only for rare and special occasions where a game achieves the kind of quality seen as appropriate for premium commercial grade games.

A game can be scored outside of these norms, even if it generally falls under one, if it has a specific feature or quirk that stands out, but that's the exception and not the norm. One exception I think is easily given is one of trying to encourage new developers, scoring their first efforts more generously and using that review and score not so much to showcase actual quality of a game, but to highlight the effort of finishing a project for a new developer. Fun fact: my first game has this exact panel review: "A good first effort by a new game author." 3 cups.

This whole "3 cups is average" conversation reminds me of some internet outrage of old, where a Nintendo Zelda game was reviewed as a 7 out of 10 and some people were furious at the "low score". It's not a low score, it's average, the average just happens to have crept up in a nominally 0-10 scoring system over time.

[lapsking]

Since reviews are always subjective, when using a 5-star (5-cup) system of grading it seems pretty human and natural for a lot of games to end up with a score of 3. I feel this mostly comes down to reviewers, whether it be the AGS panel or individuals, being polite. A 3 is a solid effort that doesn't stand out. A 2 would be a weak performance but the reviewer felt it had some merit, while a 4 is exceptionally good. A score of 1 is usually reserved for either personal issues (this game has X and I don't like X so I'll give it a 1 out of principle) or in the case of the panel reviews a genuine belief that the game lacks any considerable effort or skill in its makeup. Meanwhile a score of 5 is practically a superlative, reserved only for rare and special occasions where a game achieves the kind of quality seen as appropriate for premium commercial grade games.

A game can be scored outside of these norms, even if it generally falls under one, if it has a specific feature or quirk that stands out, but that's the exception and not the norm. One exception I think is easily given is one of trying to encourage new developers, scoring their first efforts more generously and using that review and score not so much to showcase actual quality of a game, but to highlight the effort of finishing a project for a new developer. Fun fact: my first game has this exact panel review: "A good first effort by a new game author." 3 cups.

This whole "3 cups is average" conversation reminds me of some internet outrage of old, where a Nintendo Zelda game was reviewed as a 7 out of 10 and some people were furious at the "low score". It's not a low score, it's average, the average just happens to have crept up in a nominally 0-10 scoring system over time.

Maybe AGS should also try to shift to 0 - 10 cups. Or add half a cup at least. I don't think the discussion is very complicated. But thank you WHAM, for not getting the point.  Don't be embarrassed, nobody got the point. People involved with AGS are very giving people, all they think about is how to improve our games, so giving that when you want to improve them they sacrifice their own interests and totally ignore or even deny it. The world would be much more beautiful if everybody was like them, or maybe they are! Including me? Are these people married to AGS? I'm already married and have no intention for a second wife, chill out guys. If I keep my first wife, I've done a great job. Keep your lavish heart and shiny badges to yourself, it seems they bring attachment. Do you have a shit badge for me by any chance? Something like a poo jumping up and down.

[Babar]

This specific case aside (because I have played neither game), I do think there is a bit of a problem with the Panel ratings (that are showcased here fairly well):
They are untransparent.

Panel ratings used to be quite transparent.
Obviously, every panel member had their own take on the guidelines, but it generally followed that pattern ^

It's been some years now that those guidelines have not been used any longer.

[lapsking]

It's not just the rating, but also the review. Who wrote "It's not the highest art"? What the F? Did I claim I'm making a game with highest art? Do they write this nonesense for the other games they review with 3 cups? Couldn't that Art Critique who doesn't dare to be transparent just wrote it's a good art. Did I expect them to say my art is better than Van Gogh? Or exceptional? To be honest I felt some politics in AGS from the beginning. I understand friendly "banters" but when you want to write a review please don't involve your nepotism or prejudice, or at least leave an option for a free game not to have AGS panel review, otherwise enjoy calling your reviews ridiculous. I mean just because I painted every cm with hand does it mean I expected to have the highest art? What kind of shitty review is that? I don't know exactly what I'm writing, to whom it concerns look in-between lines and see what's useful for you to improve yourself if it makes sense to you. As I do the same

[lapsking]

Not highest art but put much effort in it to me means I have been wasting my time and I could do the same thing with photoshopping online pictures and get the same review. I wasn't wasting my time, these guys are wasting their time and reputation to convince me I was wasting my time.

[WHAM]

The reviewer writes to inform the general audience of their views and opinions. Whatever the game developers intent was or wasn't doesn't really factor into that. If the reviewer feels that the audience might be looking for something (High art? Ultraviolence? Addictive gameplay? Infinite replayability? Whatever, it doesn't matter) and won't find it, then informing the audience of that can be useful. The reviewers own views and expectations will always colour the final opinion and wording, at least for as long as we have volunteers spending their own time and effort to provide these scores and reviews, and the community don't pool up money to hire paid reviewers who follow strict technical guidelines and absolute professionalism (insert your own joke about the integrity of modern day paid-to-work games journalism here).

As for increasing the number range of reviews, I don't see much point. It'd throw off all existing reviews since the values wouldn't convert 1 to 1 and, as noted in my previous message, that just leads to a new "average" score emerging arbitrarily, and then we'll just have someone else complaining that their piece was rated "only a 7 out of 10" while a game they subjectively see as less valuable or less good was given the same or similar score and they then feel upset.

The scores provide a general guidance, and as the old guidelines state: a game with 2 or 3 cups is worth playing. This implies that a 4 or a 5 is a standout and a score of 1 is reserved for games that suffer from some major issues or shortcomings that make it too difficult to recommend.

[lapsking]

The reviewer writes to inform the general audience of their views and opinions. Whatever the game developers intent was or wasn't doesn't really factor into that. If the reviewer feels that the audience might be looking for something (High art? Ultraviolence? Addictive gameplay? Infinite replayability? Whatever, it doesn't matter) and won't find it, then informing the audience of that can be useful. The reviewers own views and expectations will always colour the final opinion and wording, at least for as long as we have volunteers spending their own time and effort to provide these scores and reviews, and the community don't pool up money to hire paid reviewers who follow strict technical guidelines and absolute professionalism (insert your own joke about the integrity of modern day paid-to-work games journalism here).

As for increasing the number range of reviews, I don't see much point. It'd throw off all existing reviews since the values wouldn't convert 1 to 1 and, as noted in my previous message, that just leads to a new "average" score emerging arbitrarily, and then we'll just have someone else complaining that their piece was rated "only a 7 out of 10" while a game they subjectively see as less valuable or less good was given the same or similar score and they then feel upset.

The scores provide a general guidance, and as the old guidelines state: a game with 2 or 3 cups is worth playing. This implies that a 4 or a 5 is a standout and a score of 1 is reserved for games that suffer from some major issues or shortcomings that make it too difficult to recommend.

No, you still don't get the point WHAM, if the order is not the highest art, the will is not even the lowest part. As someone who wastes his life to make a game, and I'm forced to accept my lowest art is the same as my better art, maybe then I'm in a wrong place, which is the most possible. If you can't write an objective review don't bother with writing a review, just ignore the game. I think I know about art more than whoever untransparently wrote a review about my art. If it was New York Times it didn't matter mocking my art, it could be even a privilege. But as "community" I want to know what the F is going on here.

[WHAM]

I feel your mistake here is believing the the score range of "3 cups" magically makes the two games equally valuable, which is not the case.

And to remind you: while you and I 'waste our lives' to make games, so do the volunteer reviewers 'waste their lives' playing these games, assigning scores and writing short reviews for them. I still have games here that never received a score or review.

We know what is going on here: you completed a project and released a game, and someone out there took the time to play it and review it and to assign it a score they felt best represented its general quality amidst other releases on this site at this time.
 
That effort, for you and the panel, is valuable and means something. Be happy for that!