More Panel Ratings Discussion

[ThreeOhFour]

Maybe this is going to be the reason why other engines will beat AGS in the future?
Shouldn't we change the logo?
Bad marketing concept to use an insider object as front page sign...

Other engines will "beat" ags because of a logo? I think not.

User friendliness, features and the actual games that get made with it are far more important than a mere symbol.

[Calin Leafshade]

Maybe this is going to be the reason why other engines will beat AGS in the future?
Shouldn't we change the logo?
Bad marketing concept to use an insider object as front page sign...

Thats possibly the strangest thing ive ever heard.

Most people that come from outside don't even know what a blue cup is.
I've never seen one in real life.

They dont exist.

If you bring blueness and cupness together they annhilate like anti-matter.

[Le Woltaire]

I was joking, man...

[LimpingFish]

This has just gotten silly.

How 1 man can decide which game is good and which is bad when he has a list of games to review and he has to do it quickly (because this website needs content to be more popular).

I can't fathom how you've reached this opinion of the rating process. Pick a gaming website at random (IGN, Gamespot, Adventure Gamers, etc), and you'll find reviews are usually the work of a single author. A single author with a single opinion. Most, like the AGS database, also have a player rating, because, as anyone with a crumb of common sense knows, no one opinion is ultimate.

And the panel tries to play games through to completion, barring any fatal bugs that prevent us from doing so. There is no rush for "content".

Harg, if you feel that your work is too good to face the same critiques every other game has is the database, then that's your decision. You won't find a better reception anywhere else if you continue in this manner. You've managed to insult most of us here.

ProgZmax, I was open to criticism - almost 4 or 5 years, but not anymore.

So you've reached the pinnacle of game design? This is as ludicrous a statement as it is a pretentious one. The is no ultimate point we reach were we've learned all we can. Commercial developers with 20+ years under their belt still strive to improve their games and are usually their own harshest critics.

But to basically say "I'm an artist, and you'll either like what I do, or shut your mouth!" is an astonishing response to a community that has been supportive. From giving you a forum to advertise your games, to providing technical help should you require it, the AGS community has been nothing but supportive. Your behavior not only in removing your games, but also in the manner you've conducted yourself in this thread is quite baffling.

All this could have been avoided with a short pm:

"I do not agree with the rating comments for my game Cosmos Quest III. Due to this, I have decided to remove my game from the database."

And you would have probably received a short pm in reply, asking you to reconsider, but saying the final decision was up to you.

And we all could have gotten on with our lives.

But you pulled all of your games, and made it a point to advertise this fact in the CQIII thread. Which smacks of an attention-seeking "That's SO unfair!" hissy fit; something that belongs in the playground.

This whole situation has depressed me.

[------]

If a game receives a bad critic, the developer must accept it. If the users criticize things of a system that they don't like, should not members do the same thing?  :

[Dualnames]

Hell, I see no point being such an ass, because you think so highly of yourself, when those poor guys that worked so hard on King's Quest got only 3 cups. What should they say? But see that's what you've misunderstood. 3 cups mean must play! That's a really good rating for that game. It's reasonable!

But the whole point is during the making of this game, you thought how you'd get those 4 cups you really wanted. And you lost that loving feeling! That's all have to say.
And that is explained by how the world received your games. The topics on the other cosmo quests have 16000 views each and over 35 replies. This one has 4691 and like 30 replies and it's all not for the game and wayyyyy before the game got rated for the panel. That says it. And that's the people opinions!

I might be harsh, but isn't so the truth?

[Igor Hardy]

But you pulled all of your games, and made it a point to advertise this fact in the CQIII thread. Which smacks of an attention-seeking "That's SO unfair!" hissy fit; something that belongs in the playground.

Well, it seems that Harg just left a brief note that he is removing his Cosmos Quest games without elaborating on it. It hardly "smacks of an attention-seeking "That's SO unfair!" hissy fit" and while it is unfortunate I don't understand how it is insulting to others.

What I wanted to add to the discussion is: No one can force anyone else to like something (or even play something), even if one has perfect arguments. The only solution is to create primarily for different reasons than expected reception by other people. A cold reception might take out some joy of your efforts, but no enough to stop you.

[MrColossal]

Split this for obvious reasons.

I propose people on both sides try and calm down a touch.

To all the Panel Members: Don't take it personally when someone removes their games. Let them do what they want and let it be. There are hundreds of games not removed after being rated, focus on those and pat yourselves on the back if you want. Also accept that you are in a community and people have the right in this community to criticize functions of it. If people are upset and you can't think of a way to calmly help, don't post. Especially because you are a involved in a function these forums and therefor a voice for them.

To all the others: The panel members aren't evil. They are members of this community and volunteered to do something that was established to try and help sort out the games database. It's awesome that you all are so passionate about this community that you even care about this subject. There are so many forums that are just dead dead dead because people don't have a passion for the community. The only thing I ask is to try and remain civil and not get personal. If someone is getting personal with you, try and ignore them.

To all everyone: I would recommend against telling each other to not talk about a certain topic because you are particularly sick of hearing about it. If you are sick of it, stay out of the thread.

To all the single ladies: 'sup, 28/M/NY/20 feet tall, pix pls?

[Snarky]

ProgZmax, I was open to criticism - almost 4 or 5 years, but not anymore.

So you've reached the pinnacle of game design? This is as ludicrous a statement as it is a pretentious one.

I think the thread is suffering a bit from the fact that several of the people posting don't seem to be 100% fluent in English. (I can only understand what the heck Dualnames is trying to say about 50% of the time, for example.) I'm guessing that Harg doesn't mean that his games are now above criticism, just that he's been hearing the same complaints for 4 or 5 years, that he has (in his view) addressed them without getting any credit for it, and that he no longer has patience for criticisms he doesn't consider valid. But hey, that's just my interpretation!

Along the same lines, I'm pretty sure "How many persons are in the AGS Panel?" meant how many people are involved in reviewing/rating each individual game.

While I think that the panel review process is far from perfect, any review system worth a damn is going to hand out negative reviews from time to time, and some people are going to be unhappy. Harg, your grievances suggest fixes like providing longer reviews, or have several panel members offer their individual perspectives, but how does that fit in with your assertion that the games, not the reviews, are the core content of the Games DB? (Which I agree with.)

I think deleting the games is an unfortunate overreaction. At the same time, if Harg doesn't like this community or agree with the way important functions are being run, he's well within his right to withdraw. He's hardly the first or only creator to be oversensitive about criticism, anyway!

What annoys me a bit is the attitude of some responses of "it's just one opinion, deal with it!"--like the rating featured prominently on the AGS site doesn't matter, and it's wrong to care. Of course most developers are going to care! Essentially taunting people for "being a baby" when they don't deal as well as they probably should is not very classy, and when done by panel members, it doesn't make the panel--or this community--look very good. On that I agree with Find Therma. (Bet you wish you'd stayed anonymous now, you unwitting spokesmen for the panel! )

Not that I don't understand ProgZmax or LimpingFish's impulse--I've done pretty much the same thing myself; it can be really annoying when someone takes back something they had previously shared with the community, especially if you invested your time responding to it.

[auriond]

To all the single ladies: 'sup, 28/M/NY/20 feet tall, pix pls?

20 FEET TALL WHAT

[GreenBoy]

I've come up with a solution.

It's so simple, I can't belive no one thought of it.

You ready..... It'l blow your mind.....


Give every game a 3 cup rating, then everyone's happy.  Except for the people who had a higher rating, but they're in the minority so that's ok.   

Problem solved, fools.

[Ryan Timothy B]

Give every game a 3 cup rating, then everyone's happy.  Except for the people who had a higher rating, but they're in the minority so that's ok.   

I honestly have a feeling it'll eventually turn into something similar to that, with all these reactions lately.



Wasn't that Cosmo game a commercial game?  I can understand the panel being tougher on commercial games, because cost vs enjoyment falls into the formula.

[ddq]

Everyone needs to get a grip. The rating disclaimer says itself that it's okay to disagree, but don't feel offended if the rating isn't high enough in your opinion. I can understand feeling hurt that someone didn't like "your baby," but the system is in place to highlight the really astounding games. The last thing I'd want is for someone to rate my game anything other than what they thought it deserved.

[Crimson Wizard]

I've come up with a solution.

It's so simple, I can't belive no one thought of it.

You ready..... It'l blow your mind.....


Give every game a 3 cup rating, then everyone's happy.  Except for the people who had a higher rating, but they're in the minority so that's ok.    

There's more subtle solution.
Make, say, 20 cups minimal grade and 25 cups maximal grade. In this case you'll keep general rating concept and yet make lower grades less unpleasant.

[LimpingFish]

Well, it seems that Harg just left a brief note that he is removing his Cosmos Quest games without elaborating on it. It hardly "smacks of an attention-seeking "That's SO unfair!" hissy fit" and while it is unfortunate I don't understand how it is insulting to others.

I don't think the one line assertion "Cosmos Quest I-III - The games are removed by the author." qualifies as a note, but I see your point. Removing the games because he doesn't agree with the rating is insulting to the panel member who took the time to play his game (it's not a particularly short game) and critique what he felt were it's flaws. If we waste time rating those games that are going to end up being removed from the database for no other reason than sour grapes, why should we bother at all? It's also insulting to suggest we rate games haphazardly to meet some nonexistent deadline, when the author in question didn't bother to enter into any sort of civil communication with any facet of the community before pulling his games. If he had he might have discovered that this isn't the case. I can't really find any other way to describe it, other that a hissy fit. A tantrum.

I think the thread is suffering a bit from the fact that several of the people posting don't seem to be 100% fluent in English. (I can only understand what the heck Dualnames is trying to say about 50% of the time, for example.) I'm guessing that Harg doesn't mean that his games are now above criticism, just that he's been hearing the same complaints for 4 or 5 years, that he has (in his view) addressed them without getting any credit for it, and that he no longer has patience for criticisms he doesn't consider valid. But hey, that's just my interpretation!

That's a valid point, but any such mis-communication might possibly have been avoided, if the author hadn't chosen such a counterproductive and knee-jerk approach. Personally, it says to me that he's tired of people pointing out the faults in his games. Period.

AGS PANEL is trying to tell us how we have to write the games. No way! The indie game creator is an artist not an engenier!

Imperfect English or not, that seems a pretty strong statement.

But, has anyone here actually played Cosmos Quest III? I'd like to hear some opinions from outside this debate; opinions that may or may not dispute the validity of the comment left by the panel.

EDIT: I'm using the lesser "insult", rather than "INSULT!". "Insulting" like a insolent french waiter, not "INSULTING!" like somebody pissing in your cornflakes, while shitting on your dog. Just a little clarification.

[Dualnames]

Don't I just love this community. When something is f$^&% up, someone comes and blaims my communication skills .

I think we should take this matter more lighthearted, and I apologize for probably sounding like an ASS and taking this too damn seriously.

Personally I blame AGA.

PS: I think I made some valid points back there so I'll sum up:
1)I think Harg should have taken this into pms with the panel instead of removing his games.

2)The lack of posts and interest of Cosmos Quest III probably makes the choice of two cups just. If someone can post the AGS panel rating, so people who HAVE  played this game justify on it, would be perfect.

3)I think the panel should remain as it is, and it's as most people have said, guidelines.

4)On no account should people consider that because the panel is giving a low rating, they don't count effort. They are of course game makers themselves!

[Snarky]

[On second thought, not sure how much point there was to the first bit of the post, and I actually don't want to stir up trouble just for the sake of it, so snip it goes. Hope no one was working on rating it!]

It's also insulting to suggest we rate games haphazardly to meet some nonexistent deadline, when the author in question didn't bother to enter into any sort of civil communication with any facet of the community before pulling his games. If he had he might have discovered that this isn't the case. I can't really find any other way to describe it, other that a hissy fit. A tantrum.

Someone else (Dualnames?) suggested he should have PMed you or ProgZ. OK, maybe that would have been a better way to handle it--but it's worth pointing out that the site offers no point of contact, that the info-text with the rating suggests that game makers should not entertain complaints, and that in forum comments, both you and ProgZmax (as the highest-profile panel members) have been quite hostile to complaints about panel ratings. It does not, over all, come across as an invitation to "enter into any sort of civil communication".

[Shane 'ProgZmax' Stevens]

I think you confuse hostile with assertive and to the point.  No, LimpingFish and I haven't been hostile.  Hostile would be essentially telling anyone with a complaint to fuck off and not continuing to discuss the issue.  Now, while we may get annoyed by recent frequent complaints and extreme actions taken by certain members we've never been hostile, only restated how the system works and that the ratings are subjective, and most importantly, not to be taken seriously.  Certainly not to the point where you feel your work is being insulted and/or discredited. 

Now, I could see someone taking that sort of view with someone of influence and power like John Carmack or Jane Jensen, but generic, faceless forum members?  We're really not important enough to place on such a pedestal, nor do any of our ratings (panel or otherwise) amount to a hill of beans in the greater scheme of things.  They are merely there as suggestions and opinions by people who have played the game.  If anyone is going to be swayed by one person's opinion then I can honestly say you don't want that person playing your game anyway, because if they are interested in what you've got they'll play it regardless and draw their own conclusions.

And on the topic of being annoyed, one of the key reasons is the accusatory tone in posts like yours, Snarky.  You go from accusing the panel of potentially hiding something by not offering 100% disclosure to accusing the two panel members willing to actually discuss the ratings and their significance of being hostile.  None of your accusations are either helpful or constructive and none of them help to make any of these discussions go any smoother.  Stop trying to dredge up conflict with charged statements (this applies to everyone) and come up with something that actually benefits the discussion so that it can move forward rather than grind its wheels in 'you're this' or 'the panel's that'.

If any of you have some sober questions about the panel, by all means private message me.  There are certain things I cannot discuss, like why a rating was given, but I can explain some of the rationale that goes into the ratings.  Also, the list of guidelines will be made available soon so you can see what the panel uses as a reference when playing each game in the database.  I stress reference because people tend to take lists as a literal interpretation of what's going on and personal reactions to a game play a large role in whether or not it will be well-received.  Hopefully that makes sense!

Finally, my primary interest right now is to try and help Harg see that the rating isn't a personal attack on him or his game, nor is it of any real value.  The panel member who played it was not bowled over by the game and listed his/her reasons in the comment, and that's all.  Please don't take these ratings seriously, guys.  Trust me when I say there is no malice in any of the ratings offered, and if that time ever comes it will be dealt with severely.  We're not doing this to discourage anyone, and by and large I think we've succeeded in that.  There's a clear vocal minority who are displeased with their ratings, but there's very little that can be done to make them happy except give them a different rating, which is disingenuous.

Would it help if a person upset by their game rating was given a more detailed account of the rationale behind it, delivered to them by myself (or LimpingFish) via private message?  Would that be something people might appreciate in the future?  Granted, it's not going to help everyone but I suppose it might help some people?

I'm sure the panel judge who rated Cosmo Quest III would be more than willing to write up a couple of paragraphs on what he/she liked and disliked about the game and how he/she came to a final rating if you're interested, Harg.

What do you think?

[Layabout]

Ratings are, and always have been a flawed system of telling people whether a game/film/whatever is good. Good, fun and entertaining is too subjective an opinion to be rated in any way. I guess a more appropriate way would be having 3 cups being the AVERAGE since it is in the MIDDLE. Shit games and awful games (like my masterpiece Gorthor) deserve 1 and two cup ratings. But ratings themselves can be confusing, which I think is why people are having an issue with it. Some ratings consider anything around 50% to be average. Some others consider 70% to be average. It's not an accurate way of telling whether a game is good or not. Do away with the whole thing and perhaps include a small review only. You can still have a list of highly recommended games.

Why else would a highly flawed game like Limey Lizard get such a high rating? In my opinion, the radical user interface detracted from the playing experience, and a walking dead made me rage quit. But that game is rated 4 cups. The rating system is flawed.

I don't really think it is appropriate for such a highly established and recognised community member to be saying 'fuck' in every sentence either, especially when you are trying to argue your point.

[Snarky]

I think you confuse hostile with assertive and to the point.  No, LimpingFish and I haven't been hostile.

Let me illustrate. I think comments like these indicate that you are hostile to having people contact you with complaints:

Even from the start of the panel ratings there have been two types of people:

1.  Sour grapes posters who don't like their rating or ratings on games they like.

Frankly, I really don't care to enter into discussions with creators, simply because I don't think they can say anything that would change my mind. If you have to explain why your game deserves a higher rating, you're already wasting your time. And mine.

Let me make on thing clear, for future reference. Any ratings bullying, passive-aggressive or otherwise, will not be tolerated. If you have a problem with a rating, either leave a comment on the game's database page, in the game's official forum thread, or, if you must, PM myself or ProgZmax. [Snarky's emphasis]

And wrt hostility to criticism in general:

And on the topic of being annoyed, one of the key reasons is the accusatory tone in posts like yours, Snarky.

Those and other comments along those lines sound to me like a hostile attitude.

Hostile would be essentially telling anyone with a complaint to fuck off and not continuing to discuss the issue.  Now, while we may get annoyed by recent frequent complaints and extreme actions taken by certain members we've never been hostile, only restated how the system works and that the ratings are subjective, and most importantly, not to be taken seriously.  Certainly not to the point where you feel your work is being insulted and/or discredited.

By "hostile to" I just mean that you are opposed to it, you do not welcome it, and you give every indication that it annoys you.

And on the topic of being annoyed, one of the key reasons is the accusatory tone in posts like yours, Snarky.  You go from accusing the panel of potentially hiding something by not offering 100% disclosure to accusing the two panel members willing to actually discuss the ratings and their significance of being hostile.  None of your accusations are either helpful or constructive and none of them help to make any of these discussions go any smoother.  Stop trying to dredge up conflict with charged statements (this applies to everyone) and come up with something that actually benefits the discussion so that it can move forward rather than grind its wheels in 'you're this' or 'the panel's that'.

The point I was trying to make in my latest post was that there's a disconnect between what people are saying disgruntled game makers should do and the instructions and signals the game makers actually get. Want the implicit "helpful and constructive" suggestion made explicit? CHANGE THAT!

I also ended my involvement in the last round of argument around this with an explicit list of constructive suggestions, so I take exception to your characterization. We could go round and round on who's been most confrontational [examples deleted for the sake of diplomacy], but instead let me say that you seem to have taken offense at the word "hostile", so would you prefer "unreceptive" instead?

As for what should be done about situations like this, I pretty much think: nothing. People were deleting or threatening to delete their games from the db and their posts from the forum following criticism or dispute before the panel existed, and will no doubt continue to do so whatever system is in place. The best way to deal with it, I think, is to respond with equanimity, and gently try to convince the person to reconsider, and if that fails just let them do what they want and not worry too much about it.

But sure, opening lines of communication so people can blow off steam before it comes to such radical measures is certainly worth a try.