As some of you will remember, about a year ago a new project was launched. Its goal, to review and rate all the games in the AGS Games database.
History
Over time there have been several requests to sort games by rating on the games page. But this feature has not been added because many people felt that some games were unfairly rated and there was evidence of cheating by a few people to bump up their game's rating.
But it was clear that people new to the website needed a better way to find high quality games to play.
The Project
We assembled an anonymous group of AGS Forum users, known as the Review Panel, whose job it was to play through all the games in the database and try to give them an objective rating.
As the project got started, we realised that since we would be playing through all the games, it was a good opportunity to record some extra information about them. As a result, games have now been given a genre, and any broken download links have been marked as such. Additionally, we can now separate out commercial games from free ones.
Changes to the Games Page
You'll notice the following changes:
* The "Browse Games" section on the main Games Page has had a makeover. Rather than just browing by game length, you can now customize it to provide more useful results.
* Each game now has a Panel Rating as well as the existing User Rating.
* Each game now has genre information, and any broken links are marked as such.
Which games are not included?
Demos and games with broken links have not been rated by the panel.
Is it our policy not to rate demos, since we don't want to unfairly bias the reputation of the final game just based on its demo.
Games with broken links will be rated if the link is fixed by the game author.
My game rating is too low!
Game X has a rating far too high!
You guys are stupid, game X is way better than game Y!
Whilst the panel has tried to be as objective as possible when reviewing games, at the end of the day we're all human and we're all likely to have different opinions.
In due course we will put in place a formal appeal procedure that you will be able to use if you feel your game has been unfairly treated. In the meantime, you're free to discuss the ratings in this thread and argue about them; and if the Panel read any convincing arguments they may update the relevant rating.
But please, don't take this too seriously! We're never all going to agree on the right rating for each game, but hopefully they are consistent enough to be helpful to players. And remember, this is what this whole thing is about -- making it easier for people to find games to play.
Conclusion
First, I'd like to thank the Review Panel, who have dedicated a lot of their time over the last year to getting all these games rated. Without you guys, this project would never have got off the ground.
And secondly, I hope you all find this new Games page to be useful, and that it's been worth all the effort!
Enjoy:
http://www.adventuregamestudio.co.uk/games.php
Awesome.
One little suggestion: a check all box would be nice, so we can narrow our searches faster.
Having served on the rating committee for about 3 seconds ages ago, I realise that it is actually very tedious work sometimes getting through those games: so huge kudos and respect to all those who did this monumental task.
Very nice upgrades overall; thanks to all involved :D
Excellent work, and what a massive amount of effort this must have taken.
I've done some searching and browsing around and have to say that the ratings seem very, very fair to me from what I've seen.
Thanks for the time and effort, all involved :D.
thanks, panel, looks great, and thanks for the generous cup rating. ;D
Our goal was above all else to be fair while creating a system where the lowest rating meant the game was seriously flawed or showed little effort and the maximum cup rating for games that were (or were very close to being) perfect. This is why there are so few 5's and many 1's, because being completely honest means realizing that a great number of the submitted games were jokes/not really serious efforts. We definitely worked to give games that showed serious effort a fair shake, but in the end it all comes down to the likes and dislikes of the individual judges and the result is no different from what you would see at any rating site.
I'm glad that at least some of you so far are finding the ratings to be more reasonable than not.
Well done, everybody who worked on this.
Awsome! Really good job! :D
edit:
I wonder if they actually played my Evul click game to rate it. The link is death for more then a year now. I guess it's just awarded one cup from the silly name and screenshot :P
Well done. Thanks for all the work.
Like JBurger said: A all check box would be nice.
I think I found two unmarked games with broken links: Anna and Spooks.
Ben Jordan 4 seems to have no rating at all?
Quote from: matti on Thu 12/06/2008 09:55:42
I think I found two unmarked games with broken links: Anna and Spooks.
Both games (along with Linus Bruckman, Heartland Deluxe, and La Croix Pan) are hosted on my site, which is working for me. If the direct download doesn't work, try clicking "go to site" and then download from the link on my site using "right click, save as."
EDIT: Actually, yeah, something weird is going on with the .rar downloads when using the link from bigbluecup. Going to my site and manually downloading still works though. I try to see why this is happening.
Okay, thanks, Vince. ;)
Quote from: SSH on Thu 12/06/2008 10:10:00
Ben Jordan 4 seems to have no rating at all?
I think person who made that Ben Jordan podcast did that rating ;)
http://www.adventuregamestudio.co.uk/games.php?action=detail&id=513
You don't see the 3 cup rating? I do.
The panel hates my games, woo! :=
Apart from that, I like the update.
The Inquisit... The Panel hate my games too!!!
Nice work guys! It must have take a lot of work.
I checked the ratings for some games and I think they are pretty fair and just.
Quote from: ProgZmax on Thu 12/06/2008 10:59:53
You don't see the 3 cup rating? I do.
Darn proxy cacheing... D'oh!
Meanwhile, how long has the umlauted-you in http://www.adventuregamestudio.co.uk/games.php?action=detail&id=584 been broken? Any chance of that getting fixed?
Soviet Untergunterbuntersdorf has also got something going on. Has the codepage on the games DB changed or something?
Also, how the heck did Princess Marian XI get two cups? It's clearly a one-cup game.
This seems very well done and very user-friendly. I think many people will be very grateful for all of your hard work!
I skimmed over the ratings. Of course there will always be personal preferences there, but I actually found myself agreeing with almost all ratings for the titles I instantly recognized! For those I'd personally give higher (Prodigal) or lower (Other Worlds), I could imagine the average rating being a more likely standard.
Thumbs up!
(Although... only three cups for Adventures in the Galaxy of Fantabulous Wonderment..? I would say that this game's unique, innovative gameplay alone should make for an extra cup...)
Congrats guys on finishing this huge project!
The 'search on genre' option is a great addition, and the independent rating a very very welcome new feature.
(And thanks for the generous rating of my game ;))
I believe I found a minor bug: the "game length" pulldown is no longer available when submitting or editing a game entry.
That's a pretty big bug! I'm sure CJ will fix it asap.
Good job, panel! And nice improvements feature-wise, also.
Personally, I prefer games that break the mold a little bit more than some of the top-rated ones, but overall it seems pretty fair (though ultimately still quite subjective). This will definitely make me sit down and try the four-star (and one or two five-star) games I haven't played.
It would be nice to be able to search for specific star ratings, not just "X and higher". Sometimes, you just want to find the worst game out there.
About Paul Quest...
Quote
The AGS Review Panel has left a comment about this game:
Extremely shoddy. Download links to RapidShare.
I really feel a little hurt. I understand the rating, but not the comment. Anyway, thanks for the time you guys take to play the game. And a good job with the games page.
jp
Good idea, though I didn't get the ratings I hoped for lc3 anyway, have to admit the rating is objective and maybe lc3 had to get that vote. Anyway, it was a huge project and glad they actually played my games, hopefully they didn;t bother them a lot..
Having checked Nelly Cootalot's rating, I entirely endorse this new system. Hurrah for the new system!
Can I suggest that an AGS Review Panellist rating for each game would be a nice addition eventually. That way players could guage their tastes against the reviewers. I find it useful to look at the reviewers' names on Just Adventure because I know that I don't share some of their tastes, so I can take their ratings with a pinch of salt.
Quote from: Radiant on Thu 12/06/2008 21:30:19
I believe I found a minor bug: the "game length" pulldown is no longer available when submitting or editing a game entry.
Oh yeah sorry, I forgot to mention this.
We also decided that because everyone seemed to have their own opinion on what a "Medium-length" game was vs a "Full length" one, new games submitted from now on will not allow the author to set a length. Then, when the Review Panel reviews the game, having played it they will be in a good position to set the game length at that time.
Also, it allows "my first game"-type games to be moved into the Training Game category where the game authors may not allocate it there themselves.
Quote from: JpSoft on Fri 13/06/2008 12:55:27
About Paul Quest...
Quote
The AGS Review Panel has left a comment about this game:
Extremely shoddy. Download links to RapidShare.
I really feel a little hurt. I understand the rating, but not the comment.
Apologies, that was only to serve as a note for the reviewers comment. The proper comment has now been added.
Quote from: Ali on Fri 13/06/2008 15:42:28
Can I suggest that an AGS Review Panellist rating for each game would be a nice addition eventually. That way players could guage their tastes against the reviewers.
To be honest, the original plan was to keep the rating panel anonymous. I'm not sure how knowing a reviewer's taste would somehow lessen the validity of their review, though. It certainly didn't influence their ratings.
Quote from: Pumaman on Fri 13/06/2008 18:18:42
We also decided that because everyone seemed to have their own opinion on what a "Medium-length" game was vs a "Full length" one
Ah, that makes sense. People can always use the vote section to point out that "this game belongs in that other category".
I'm curious, is the rating of Diamonds in the Rough based on the demo or the full game? I'm only asking because if it's the former, I will be happy to give you guys the full game for your review.
Can I search games by date?
Wow, congratulations on this massive job. I know what it's like because I'm on my way playing them all as well. But I hunt the broken links and search beyond the DB.
That brings me to the following. Since there will be a panel judging the games and thus the pages are moderated, is it an idea to make the adding games to the database less voluntairily? With that I have the idea that if a game is not added by the author within a week of completion/announcement, the game will be added by a mod. The password of the page will then be pm'ed to the author so he/she can update it to their liking.
This will result in the advantage that games left in the Completed Games Announcement, HOUR games, The MAGS competition or the OROW competition (or maybe any other in the future) are still added to the DB and thus creating a more complete overview of games. If the author doesn't want to change the entry, fine but it's still documented/registered and won't drown eventually in threads, maybe never to be dug up again.
It's just a thought...
No. The database is a purely voluntary effort for people who want their games to be there. We're not going to start forcibly adding games that people might not even necessarily want a large group of people to know about. Voluntary also means they are signing on willingly for the user/judge ratings; adding a game without the author's knowledge would be a different issue.
I think it's reasonable to assume that people who make an AGS game know about the database, so they consciously do not submit their game.
My first MAGS game isn't in the database, because I consider it a failed experiment.
Quote from: Leon on Sat 14/06/2008 23:40:27
This will result in the advantage that games left in the Completed Games Announcement, HOUR games, The MAGS competition or the OROW competition (or maybe any other in the future)
MAGS has its own database which does include all MAGS games to date; Hour and OROW games are in the forum, which technically is also a database. RON also has its own database on its forum.
I'd like to have as complete a list as possible of AGS games here at bigbluecup, and I'd strongly encourage everybody to add their games to it; but if people for whatever reason don't feel like doing that, it's their choice.
Great job -- the additional features definitely improve browsing the games section, and implementing the new rating system was a good idea as well.
Quote from: ProgZmax on Sun 15/06/2008 00:00:19
We're not going to start forcibly adding games that people might not even necessarily want a large group of people to know about.
Maybe adding the options "Make only visible to members" or "Do not publish executable"?
Quote from: ProgZmax on Sun 15/06/2008 00:00:19
adding a game without the author's knowledge would be a different issue.
I never said that. The author still has influence on it. The author will be send the password to update their page.
Quote from: TwinMoon on Sun 15/06/2008 00:04:23
My first MAGS game isn't in the database, because I consider it a failed experiment.
You're fully entitled to that, I know. And I respect that. But I'd still like to know about it. I'd like to see how you progressed. And what you think of as 'failed' might not be by me. What people consider Joke games aren't to me and games I find hilarious, are drama to others. It's all a matter of interpretation and perception.
Quote from: Radiant on Sun 15/06/2008 00:23:00
MAGS has its own database which does include all MAGS games to date
MAGS is in that respect the only really complete database because..... you're entered automatically (competing is entering). Why not apply that view onto the whole of AGS and say: publishing is adding to the DB?
Quote from: Radiant on Sun 15/06/2008 00:23:00
Hour and OROW games are in the forum, which technically is also a database.
Speaking of the forum as techinically a database well.... yes but then the whole internet is a database. I don't want to go through all threads to find the game that got lost in publication. I want to find the games that I'm not aware of, not the games I know are there.
Quote from: Radiant on Sun 15/06/2008 00:23:00
RON also has its own database on its forum.
And RON also has it's own DB and yes, these are also administred better but that's because it's part of the game, otherwise you wouldn't be part of the 'story' RON is about. (But when your game gets rejected you get a complete different discussion)
I completely understand what you're all saying and to some extend I agree. There should be freedom for the author. But what I'm having difficulties with is the administration and maintenance. The addition to the DB is based on voluntary grounds, there is minimal moderation and there's little maintenance. There are enties both in MAGS and in the games DB, or in RON and the games DB or there are games that are in neither. A lot of links don't work anymore and like said before, I think it's incomplete.
If things are that voluntary why aren't entries deleted once the links are down for a longer period of time? What will be the use of the page if the link doesn't work (anymore)?
You must all understand that I see a huge difference in administering the publication of a game and the possibility to play/download. Sure, the optimal situation would be both but if the author doesn't want me to play his game that's fine, that's his choice. Then I should have downloaded the game when it
was available. But at least I know the existence of the game and I can contact the author about it. See it as publishing books. You might choose not to print any more issues, but you can't say to people: "I want my book back". The bookshop can still tell me that you wrote it once and you're not printing it anymore. Then that's my bad luck. Then I can only ask around or contact the author if he's still got a copy (if I'm that desperate). And if I don't like the entry in the DB, I won't bother. But at least I knew of it's existence.
Quote from: Leon on Sun 15/06/2008 10:04:02
If things are that voluntary why aren't entries deleted once the links are down for a longer period of time? What will be the use of the page if the link doesn't work (anymore)?
Correct me if I'm wrong, but I believe there is an ongoing effort to locate games with dysfunctional links, and host them on americangirlscouts.org. That might help.
According to the database, there's only 26 games with no functional download. (edit) Wait, I can't count. Make that
152 (26 of which have a rating of 2+)
(edit) speaking of which, perhaps somebody from the rating team can post a list of all games that lack downloads in the forum, then the community can probably locate half of those instantly on their hard drives? :)
Quote from: Radiant on Sun 15/06/2008 11:04:22
I believe there is an ongoing effort to locate games with dysfunctional links, and host them on americangirlscouts.org. That might help.
That would be an improvement, and only part of the discussion. The main object here is to create a database with all published games. With monitored links. If the author doesn't want to publish his/her game, state so on the page. If they don't mind, the link should be maintained and monitored. Moving the executable to a separate host for easier and better control would even improve it more. [edit] But when you do that, the author loses control.
Quote from: Radiant on Sun 15/06/2008 11:04:22
Make that 152
That's more than 15%. Quite high I think. And then I even included the demo's and the non-adventure games.
[edit]
Quote from: Radiant on Sun 15/06/2008 11:04:22
(edit) speaking of which, perhaps somebody from the rating team can post a list of all games that lack downloads in the forum, then the community can probably locate half of those instantly on their hard drives?
Aren't you then doing exactly what I'm suggesting? Keeping track and control over published games? It's pretty much like moderating your DB.
Quote from: Leon on Sun 15/06/2008 10:04:02
You're fully entitled to that, I know. And I respect that. But I'd still like to know about it. I'd like to see how you progressed. And what you think of as 'failed' might not be by me. What people consider Joke games aren't to me and games I find hilarious, are drama to others. It's all a matter of interpretation and perception.
There's a difference between a bad game and a joke game.
Grundislav's "Porn Quest" might arguably be a bad game (only one puzzle) but it's a great joke game (the joke being that it ends when you get the porn).
However, you convinced me. I'll put it in the database, mentioning I consider it a failed project.
Quote from: Leon on Sun 15/06/2008 10:04:02
Quote from: Radiant on Sun 15/06/2008 11:04:22
(edit) speaking of which, perhaps somebody from the rating team can post a list of all games that lack downloads in the forum, then the community can probably locate half of those instantly on their hard drives?
Aren't you then doing exactly what I'm suggesting? Keeping track and control over published games? It's pretty much like moderating your DB.
It's more cleaning up the DB. Those games were initially added to the DB. And I second Radiant's idea. Post the games with broken links in the forum. It's no use having games with broken links in the DB.
Well done for finishing off this huge task Peeps
Quote from: Radiant on Sun 15/06/2008 11:04:22
According to the database, there's only 26 games with no functional download. (edit) Wait, I can't count. Make that 152 (26 of which have a rating of 2+)
The broken links are a problem, and one the panel is looking into. In fact, a number of links for games that were rated almost a year ago, have since expired (usually those hosted on sites with 30/60/90 day inactivity clauses).
Finding missing games, outside of the the developers themselves repairing their links, will take time. As for Bicycles-for-Slugs becoming the main repository of working AGS downloads, this is still the plan.
Like other panel members have said in the past, having a fully functioning and up to date database is, and will be continue to be, an ongoing job.
And thanks for the feedback, everybody. :)
Great work, people. Nice to see the community in action again :)
Quote from: Leon on Sun 15/06/2008 10:04:02
Quote from: ProgZmax on Sun 15/06/2008 00:00:19
adding a game without the author's knowledge would be a different issue.
I never said that. The author still has influence on it. The author will be send the password to update their page.
But it would still be adding the game to the database without thier agreement to it. And I think adding every game that pops up here and there to the DB would be both silly and painful. It works just fine as it is.
Quote from: alkis21 on Sat 14/06/2008 00:22:11
I'm curious, is the rating of Diamonds in the Rough based on the demo or the full game? I'm only asking because if it's the former, I will be happy to give you guys the full game for your review.
This does raise an interesting question about whether the panel should be rating commercial games or not.
What do people think about this? Should the panel rate commerical games, and if so should they be rated against the free games or on a different scale, comparing them with other commercial titles? Or should the panel simply not rate commercial games?
Quote from: jetxl on Sat 14/06/2008 14:53:37
Can I search games by date?
Well, you can get a date-ordered list here:
http://www.adventuregamestudio.co.uk/games.php?action=digest&sortby=2&sortdir=1&startfrom=0
if that helps.
Quote from: Ishmael on Sun 15/06/2008 19:18:06
And I think adding every game that pops up here and there to the DB would be both silly and painful. It works just fine as it is.
I don't see the point. What's silly about administering games? What's so painfull about it? I think that a complete overview of games is an addition to the site. Just a bit more moderation and a bit less freedom to the author that's all. That keeps your database clear, up-to-date and userfriendly. But appearantly I'm alone in this so I'll leave it and keep my own administration.
Quote from: Pumaman on Sun 15/06/2008 20:22:25
Quote from: alkis21 on Sat 14/06/2008 00:22:11
I'm curious, is the rating of Diamonds in the Rough based on the demo or the full game? I'm only asking because if it's the former, I will be happy to give you guys the full game for your review.
This does raise an interesting question about whether the panel should be rating commercial games or not.
What do people think about this? Should the panel rate commerical games, and if so should they be rated against the free games or on a different scale, comparing them with other commercial titles? Or should the panel simply not rate commercial games?
Link to the relevant Adventure Gamers review ;)
I would say that if a game is listed anywhere else than in the demo section (where the review is obviously of the demo), the rating should definitely be for the full game. I surely expected the scores for the Blackwell games to represent the games and not just the demos since they were listed under medium length.
I don't think commercial AGS games need to be compared directly to mainstream commercial titles, but the price must be taken into consideration. I review a lot of indie games, and it's really not that difficult to evaluate the quality/price ratio. At least to me it comes quite natural to think stuff like "This is a great game, but I would never pay 30 bucks for it. At $15 it would have gotten a full score."
Quote from: Pumaman on Sun 15/06/2008 20:22:25
What do people think about this? Should the panel rate commerical games, and if so should they be rated against the free games or on a different scale, comparing them with other commercial titles? Or should the panel simply not rate commercial games?
In my opinion, I see no compelling reason why the panel would avoid rating commercial games, so by all means rate them. And we've had, what, four commercial games throughout the history of AGS? So I don't see why they should be an exception.
As for on what kind of scale - I'm in favor of using one and the same scale for everything, since that is the only way to have a clear outcome. However, this depends on what the panel has been doing for other sub-selections. If, say, MAGS games have been rated on a different scale than, say, Full-length games, then commercial games also deserve a different scale (because people would have higher expectations of them).
Quote from: Radiant on Sun 15/06/2008 21:45:59And we've had, what, four commercial games throughout the history of AGS?
Quite a bit more than that, actually:
Adventures of Fatman
Force Majeure 2: The Zone
Hauntings of Mystery Manor
Yahtzee special editions (5 games)
The Shivah
Al Emmo and the Lost Dutchman Mine
Intrigue at Oakhaven Plantation
Blackwell Legacy
Super Jazzman
Banana Man
Blackwell Unbound
Diamonds in the Rough
That makes 16. Did I forget any? I think they're more or less listed in order of release. And there's both Resonance and Blackwell Convergence coming up (what's next, Blackwell Synergy (http://www.blackwell-synergy.com/)?). I think the amount of commercial AGS games is quite considerable. And that's also why there's an "exclude commercial games" checkbox in the games database. So, seeing as a user can choose not to see commercial games listed, it means that they should be treated like any other game in the list for those who choose to include them. But it would be good if the developer could link to any external reviews in the game desciption (or to press clippings on own website), since it's difficult to decide to buy a game based just on a cup-rating.
Quote from: Radiant on Sun 15/06/2008 21:45:59In my opinion, I see no compelling reason why the panel would avoid rating commercial games, so by all means rate them.
I agree.
Quote from: AGA on Sun 15/06/2008 20:39:17Link to the relevant Adventure Gamers review ;)
I don't understand, are you saying that the rating of DITR is simply a copy/past of the Adventure Gamers grade? If so, what made them choose that particular review and not one of the other 14 that are currently online?
Note that I'm not complaining about the grade or anything, just trying to figure out whether the game was actually played by the people who rated it, or whether they only played the demo, or simply accepted the AG review.
Personally, I'm against rating commercial for a number of reasons that I've already discussed with the rest of the panel. I'm of the opinion that mixing commercial and free ratings, based on the same list of rating criteria, simply won't work.
For instance, rating Blackwell Unbound 4 cups and Trilby's Notes 5 cups. Is Trilby the better game, or does it rate higher because it's free? Would Blackwell be 5 cups if it was also free? Would we have to "punish" an otherwise good game because we don't think it's value for money?
I've been of the opinion that if the entries for commercial games in the database link to demo versions, they should be classed as demos, and should therefore not be rated. Or, as a compromise, label them as Full Length/Commercial games, bypass a rating (and bypass the confusion over whether we rated the full game or the demo), and rely on the developers to link to any reviews themselves.
Quote from: alkis21 on Sun 15/06/2008 22:54:40
Note that I'm not complaining about the grade or anything, just trying to figure out whether the game was actually played by the people who rated it, or whether they only played the demo, or simply accepted the AG review.
The full game was rated. One of the panel had access to a copy.
OK thanks for letting me know.
2 cents? I still think the commercial games should be rated the same as any other in the database, purely because the ones that make it into the games pages are made with AGS & home-bred from these forums & the site just like all the rest. Freeware developers are being rated & at the end of the day you don't necessarily expect that they are really the ones going out there to be rated.
Yet commercial games are there to be reviewed, hit the press & go under the hammer. Why therefore would we need to exclude commercial games from getting their review simply because they've chosen to go commercial into the big-bad-world of commercial game reviews but all the free games are the focus of opinion from the panel?
The panel, of all raters & reviewers, will probably have the most balanced & honest review of the game in hand anyway....cos we're all from AGS! I think the commercial developers should strive to achieve high cup games as would any freeware developer. It all just seems a bit backwards to me to suggest the games made with AGS you pay for would have no rating. Obviously demos wouldn't be rated but thats the same all around, isn't it? Why should there be a divide in the "all complete AGS games" part? :/
Yes, but when we're reviewing games we're putting ourselves in the shoes of the people who will potentially be downloading and playing them. We're telling them that, in our opinion, such and such a game is worth 3/4/5 cups, and that it's worth playing.
But a person who is going to get that game for free, and a person who has to pay $14.95 for the privilege, will have two very different perspectives.
It's not that the quality of commercial AGS games would be depreciated simply because they are commercial, but it does change how I would approach rating them.
Which is why I think it's a bad idea. :-\
Shouldn't there be a sort by rating option (or have I missed it)?
When you sort by rating it would be nice to have the 5 cups at the top and the 1 cup at the bottom.
Quote from: Ryan Timothy on Mon 16/06/2008 02:22:42
Shouldn't there be a sort by rating option (or have I missed it)?
When you sort by rating it would be nice to have the 5 cups at the top and the 1 cup at the bottom.
You mean, like this: http://www.adventuregamestudio.co.uk/games.php?action=digest&sortby=3&sortdir=0&startfrom=0
Although I notice that the latest games added AFTER the db update are not listed in the digest when you sort by date added...
QuoteAlthough I notice that the latest games added AFTER the db update are not listed in the digest when you sort by date added...
Thanks for spotting that -- it, along with the RSS feed, is now fixed.
QuoteLink to the relevant Adventure Gamers review
This is a good point, we've talked before about the Games page linking to reviews of the game on other websites. I guess one option would be to assume that commercial games will get reviewed elsewhere, and simply link to external reviews instead.
I think most, if not all, commercial AGS games, have been reviewed by AG. DitR and most of Dave's stuff have at least.
Edit:
Blackwell Unbound (http://www.adventuregamers.com/article/id,810)
Blackwell Legacy (http://www.adventuregamers.com/article/id,718)
The Shivah (http://www.adventuregamers.com/article/id,696)
Diamonds in the Rough (http://www.adventuregamers.com/article/id,875)
Al Emmo (http://www.adventuregamers.com/article/id,683)
Super Jazz Man (http://www.adventuregamers.com/article/id,719)
The Adventures of Fatman (http://www.adventuregamers.com/article/id,297)
I'm pretty much with LimpingFish on this, and we have discussed this matter in some detail at least twice before without a solid resolution. What we're looking for here is some kind of solidarity on the 'to review or not to review' commercial games on the basis that we will be tougher on them because they are being produced for profit, which creates a different set of expectations than a free game altogether. It seems that a good number of people are for this, but just be aware that commercial games will be regarded with far more scrutiny by the judges and the ratings may seem low compared to many of the freeware games in the database.
If this is acceptable then we will certainly continue to rate them.
Indeed. If developers are happy to have their commercial games reviewed based on an alternative criteria, as ProgZmax has said, then I guess we should do it.
All lot more would have to be considered, of course, when deciding an overall rating for a commercial game. Whereas a free AGS game might be rewarded with a 3 cup rating purely on the strength of it's gameplay, a commercial game will be judged on all aspects; from presentation to value for money. So in a sense, it will be more difficult for a commercial game to achieve a positive rating.
Developers opinions on this would be much appreciated. :)
A really simple request (which if implemented would probably actually equate to a lot of work), why not also have half cups for the AGS panel ratings? It just seems to me that the quality of the 3 cup games varies wildly, with 10 steps instead of 5 you could easily fine tune these ratings rather then having to dump all the "average" games into a single catagory.
Also as well as having the ability to search by AGS panel ratings, would it be possible to also have the option of instead searching by user ratings?
Thanks.
About the rating thing, I think the panel members had already discussed about it.
IMO, no matter how hard a panel member tries, ratings are a bit subjective, if the scale is as precise as 10 steps, I think whether a game shall get a 3.5 cup rating or a 3 depends even more on personal preference. Moreover, I think a 10 step scale is probably too complicated, as we can't tell how difference a 3.5 is from a 3 or a 4, but if we do it simple to have only full cups, the difference between a 3 and a 4 is more clear-cut.
Quote from: Gilbot V7000a on Fri 20/06/2008 07:27:49Moreover, I think a 10 step scale is probably too complicated, as we can't tell how difference a 3.5 is from a 3 or a 4, but if we do it simple to have only full cups, the difference between a 3 and a 4 is more clear-cut.
It doesn't have to be that complicated, simply keep the current guidelines but break each number into an upper and lower tier. So take all the games which recieved a 3 (A decent game, give it a go), the ones that really stand out in that catagory get a 3.5 and the rest get a 3.0. In this way it is not that much harder to rate as you are still using the base 5 catagories but we can now see which 3s were close to being 4s and which 3s were just 3s.
I really think that the 5 cups rating is fine how it is. If we're going to go into cups and a half, we'll soon probably get people saying "Why don't the panel use a percentage figure instead?".
If you want a more detailed idea of a game's quality, look at the user ratings. They're there for a reason :).
To be honest, I think 5 different ratings is too many. The difference between 4 cup and 5 cup games is marginal. We certainly don't need more.
Maybe if the five cup rating was reserved for epic games of superior quality that have left an unfading mark on the community, such as Pleurghburg, and um... eh, Ace Quest?
Or maybe not.
Quote from: SSH on Fri 20/06/2008 10:49:21
To be honest, I think 5 different ratings is too many. The difference between 4 cup and 5 cup games is marginal. We certainly don't need more.
I agree. While I see a clear difference between 3 and 4 cups, there doesn't appear to be a distinction between 4 and 5 except for personal preference, kind of like the pick of the month. This is probably because there's just a handful of 5-cup games total - less than one per year, as it stands.
(edit) I would like to speak (PM) with the person from the rating panel who writes those one-liner "official" reviews for games. While I believe the rating panel has done a good job at cup-rating the games, I find these reviews have a tendency to lack objectivity, and to be overly negative in tone. Since these are (assumedly) written by one person, why is it that this person isn't simply using the "user review" feature like everybody else?
Quote from: Radiant on Fri 20/06/2008 13:00:27
I would like to speak (PM) with the person from the rating panel who writes those one-liner "official" reviews for games. While I believe the rating panel has done a good job at cup-rating the games, I find these reviews have a tendency to lack objectivity, and to be overly negative in tone. Since these are (assumedly) written by one person, why is it that this person isn't simply using the "user review" feature like everybody else?
It depends, since whoever rated the game is the one who left the comment.
The panel comments are also voluntary, and are usually written as a courtesy. I can't say that I've seen "overly" negative comments; Indeed, the only negative comments would largely be those on one and two cup games.
On the other hand, the reasons why a game is rated three cups instead of five cups are usually all negative ones, no? So I see no reason for the reviewer not to point them out.
Why would we use the User Review function? User reviews and ratings are notoriously prone to wild fluctuations in accuracy. Panel activity should be, and is, kept separate.
Of course, all ratings/reviews are simply the opinions of the panel members and don't claim to be otherwise. When a panel member rates a game, that rating is open to objections by the rest of the panel.
Quote from: Ishmael on Fri 20/06/2008 11:29:35
Maybe if the five cup rating was reserved for epic games of superior quality that have left an unfading mark on the community...
All marks fade with time. ;)
When the panel was formed, we agreed we wouldn't just go with the popular consensus on "landmark" titles. Everything had to be rated with a fresh eye.
And since the rating system has been (and will continue to be) anonymous, we're not exactly going to tell you who rated what just because you don't agree with their assessment of your game. And I 100% disagree that the comments lack objectivity; just because some are 'negative' does not mean they were not written by someone who took the effort to play your game and weigh its merits. If they found it lacking then sobeit, as long as they weren't blatantly disrespectful or intentionally insulting. It doesn't mean the end of the world, and it certainly doesn't mean people will avoid playing it (look at the number of downloads some of the worst rated games got BEFORE the cup ratings!).
If you want to discuss a few specific comments, feel free to pm me your concerns.
I checked that there are a total of 32 games included into "training games". 31 of them have 1 cup and just 1 deserved 2 cups. Curious, at least. If some people discuss about have a different rating for ocmercial games, do not be the same for a training game? I read the thread of some of this games and a lot of them were created in 1 week, or even less than that. I downloaded some of them and im very sure that the diference between some games is more than 1 cup.
Just a free opinion.
Jp
I like this system very much.
It helped me to find and play the best games...
But is it just one person that gives the cups?
Shouldn't there be at least three opinions from different players?
It's hard enough finding one person to do a rating. If it can be organised in the future a second or third pass is an option but it's probably not happening any time soon.
Yes the ratings are subjective, which is unavoidable and a good thing in my opinion. I don't want to know what games are popular in the scene, I can see the download stats and google hits already, I see the ratings as just another source for suggestions. They do the important job of diverting people away from the games they definitely do not want to play though. I was in favour of 3 levels to start with, the 4-5 cup split was always going to cause controversy, these games being the most high profile, but 5 is what came out of the discussion and it's here to stay. Good job overall.
There are a number of members who rate different games, but yes, a game rating is usually the opinion of a single person. As are the individual user ratings, I might add.
And again, all ratings, made by the panel or otherwise, are subjective.
Quote from: JpSoft on Sat 21/06/2008 10:29:29
Curious, at least. If some people discuss about have a different rating for commercial games, do not be the same for a training game?
"Training Games" is in itself a compromise. It means the panel understands that these games were just made to try out AGS, but a cup rating will still be awarded just the same.
About two weeks ago I got some emails where several persons congratulated me.
They wrote me that Earl Bobby got five cups, and I didn't understand what they actually mean with this...
When I took a closer look at the games site I recognized the rating,
and thought it was an interesting idea to add a optional rating,
especially because many of the public ratings suffered from vandalism,
like multiple or destructive voting (there was a time where every person in the internet could vote for or against a game in the database).
When I took a closer look at the search function I soon recognized, that only five games (of 904) got a five cup rating. When I filtered the "five cup games" I saw my own game next to all those award winning games that I always admired like "Trilby's Notes" or "Apprentice". Other games like "5 days a stranger" that I expected in the five cup rating as well were just in the four cup section... Besides of that, my game was the only one in the five cup list, that didn't get an award...
I soon started to ask myself, if I really merit such a high rating with my game...
There was never really any reaction on it.
People didn't seem to like it very much and I got a lot of emails from persons who told me,
that it surely was an effort, but not really a good game.
I even tried to promote my game in other forums for the first time, just to see if someone is interested,
but even there the reactions were almost zero.
I think this shows the subjectivity of this type of rating.
I see it as a big compliment, however it fills me with subjective doubts and sense of guilt towards other games...
Ahh Le Woltaire I think it was a fabulous achievement. There will always be people looking for more but Earl Bobbys Balls a fantastic game to say the least. I was also a bit surprised that Trilby's Notes was in there rather than 5DAS, where I have read in lots of places and gotten the impression people preffered 5DAS. But, the ratings will never truly speak to everyone. The public vote won't necessarily do you any favours, and I think thats part of a reason why the cup system was introduced. Anyway, I don't think you should feel guilty...show some Balls! ;) Your achievement is recognised in a great place, if not in some of your emails!
You mentioned earlier the game length featured is now gone, the reviewer will put it in the right category.
I think this is fine for short / medium / long, but my MAGS game "Dead of Winter" has been placed in the 'short' category which I don't think is right.
Any official person who can change this for me?
Also a checkbox to state you're uploading a MAGS game would be a good idea.
Quote from: TwinMoon on Sun 22/06/2008 16:02:12
You mentioned earlier the game length featured is now gone, the reviewer will put it in the right category.
I think this is fine for short / medium / long, but my MAGS game "Dead of Winter" has been placed in the 'short' category which I don't think is right.
Any official person who can change this for me?
Also a checkbox to state you're uploading a MAGS game would be a good idea.
Dead of Winter was played in full, and "Short" was felt to be the correct category for it. It meets the criteria for such, anyway.
Categories for MAGS, OROW, Hour games, etc, were thought to be superfluous to requirements, as the game's creator can make a point of mentioning this in their initial comment when adding their game to the database.
The new database is still in it's infancy, so changes to how it works may be introduced in the future. But for now we have a workable, functioning database, so let's just see how it goes. :)
And keep giving us feedback.
Ah, okay. Thanks for clearing that up.
My confusion comes from the fact that the category 'MAGS' still exists. ( My game is a short game, no debate there, but since the database includes MAGS games as a category in length, I thought it would be put in there. )
IMHO the category "MAGS" or maybe "Games made in competitions" or something should remain since games made with a deadline are different from normal games, and people have different standards for it.
At least as an extra option (like with commercial games) so people can include/exlude them from their searches.
Actually, no, you're right. DoW belongs under MAGS, which does indeed remain as a separate category. It's OROW and such that don't have a category.
It just slipped my mind that DoW was made for MAGS.
Slight lapse of memory on my part, but I've corrected the game entry. :)
Thanks. Complaining does work ;)
Complaining makes people hate and distrust you; asking nicely often works :D.
Quote from: LimpingFish on Sat 21/06/2008 21:17:49
Quote from: JpSoft on Sat 21/06/2008 10:29:29
Curious, at least. If some people discuss about have a different rating for commercial games, do not be the same for a training game?
"Training Games" is in itself a compromise. It means the panel understands that these games were just made to try out AGS, but a cup rating will still be awarded just the same.
Soooo.... the term "training game" basically refers to someone's first game when it isn't any good?
Surely good first games are equally "training", no?
How would I as a games creator bring attention to the fact that a broken link has been fixed? I tried scanning this thread for instructions, but couldn't se any. If I missed it, please point me in the right direction.
I think this new rating is interesting, but I think the outcome will be that more people will play a smaller selection of games (i.e. the ones with many cups). Yes there is the alternative votes by people who played the game, but I can't see how this will ever have an impact for game sthat receive a low cup rating to start with as they are unlikely to be played.
I also have never understood and still do not understand, why the user rating can not be used for searching the DB. Especially now that the cups are in place, and we can search by them.
It's fun to see some change happening with the games page though! Kudos to the team of rating people for completing this daunting task!
Anyone can edit their own game's page, by clicking the "Are you this game's author? Then you can edit its entry." link at the bottom of the page. Do that if you want to correct your game's download link. You don't need a password to do this if you're signed into the forums with the same account as the game's creator is marked as.
Quote from: mätzyboy on Mon 23/06/2008 10:51:04
I think this new rating is interesting, but I think the outcome will be that more people will play a smaller selection of games (i.e. the ones with many cups). Yes there is the alternative votes by people who played the game, but I can't see how this will ever have an impact for game sthat receive a low cup rating to start with as they are unlikely to be played.
This was the issue before, with people looking for games with plenty of downloads and users' grades. Problem was that less honest authors "improved" those figures on their own. The cup system can't be abused in this way.
Instead of regarding it as a system that only promotes high grade games, you can see it as a way to ignore the games that aren't worth downloading and playing.
I think we must all acknowledge that a lot of people have published games that they didn't spend much time or serious effort in producing. A good portion of the database are very rashed releases, made only because it's cool to have a "game" out there and get some attention.
Quote
I also have never understood and still do not understand, why the user rating can not be used for searching the DB. Especially now that the cups are in place, and we can search by them.
For the reason you mentioned yourself; there's no need to further encourage people to avoid the "middle" games and only focus on the top titles.
Quote
It's fun to see some change happening with the games page though! Kudos to the team of rating people for completing this daunting task!
I agree, the panelists have done a tremendous job. As I was involved in, and partly responsible for, the conception of this project, I can only regret that I haven't been more active in the rating process.
Quote from: mätzyboy on Mon 23/06/2008 10:51:04
I also have never understood and still do not understand, why the user rating can not be used for searching the DB. Especially now that the cups are in place, and we can search by them.
If the mountain won't come to Muhammad, SSH will bring him the mountain on a silver plate: http://ssh.me.uk/ranking200712.html Horray to SuperScottishHero and handshake to everryone who worked on the cups thing.
Quote from: AGA on Mon 23/06/2008 11:00:10
Anyone can edit their own game's page, by clicking the "Are you this game's author? Then you can edit its entry." link at the bottom of the page. Do that if you want to correct your game's download link. You don't need a password to do this if you're signed into the forums with the same account as the game's creator is marked as.
The game link is fixed since a few months back and is working. What I'm wondering is if I can draw the voting panel's attention to the fact that I have fixed the link.
Quote from: Andail on Mon 23/06/2008 11:06:48
Quote
I also have never understood and still do not understand, why the user rating can not be used for searching the DB. Especially now that the cups are in place, and we can search by them.
For the reason you mentioned yourself; there's no need to further encourage people to avoid the "middle" games and only focus on the top titles.
I disagree, I think we can find plenty of examples where the user rating differs from the panel rating, and that is where the user vote becomes interesting. I would for example be interested in looking for games with a high user rating and low cup spec. It is also the only way for peer reviewers to show an opposing opinion to the panel's rating. By not allowing searching for user rating we give the cup rating validity, and disqualify the user votes completely for any useful comparison. The panel vote is thereby not questionable.
Quote from: TwinMoon on Sun 22/06/2008 22:40:12
IMHO the category "MAGS" or maybe "Games made in competitions" or something should remain since games made with a deadline are different from normal games, and people have different standards for it.
At least as an extra option (like with commercial games) so people can include/exlude them from their searches.
In my opinion, whether or not a game is a "MAGS game" would best be implemented as a checkbox (just like whether a game is "commercial" is also a checkbox).
This is because some MAGS games are short games, some are medium-length games, and some are non-adventure games; if all of those were simply placed in the category of MAGS games, it would not be possible to tell them apart.
It is useful to be able to search for MAGS games; however, I believe it is also useful to be able to search for all non-adventure games, or all short games, and so forth, regardless of whether or not these were made in MAGS.
In the essence of MAGS games being indicated by a checkbox, should OROW games be aswell? Since they're usually at the shortest end of the short games range, and not everyone knows at all what OROW is...
Mags as a checkbox instead of a category is indeed better, for the search option reasons Radiant put forth.
(and with extra option I meant checkbox of course, my mind needs a holiday I guess)
Quote from: Ultra Magnus on Mon 23/06/2008 04:28:03
Soooo.... the term "training game" basically refers to someone's first game when it isn't any good?
Surely good first games are equally "training", no?
Heartland Deluxe was not the first game I made, but it was the first game I decided to give a proper release to. In learning AGS, I had made numerous "games" beforehand that will never see the light of day. Those are
my training games.
When people choose to add this kind of game to the database (usually accompanied by "My first ever game! I made it to try out AGS! It's crap, but please play it!") they receive the label "Training Game".
Quote from: mätzyboy on Mon 23/06/2008 11:38:17
The game link is fixed since a few months back and is working. What I'm wondering is if I can draw the voting panel's attention to the fact that I have fixed the link.
Sorry about that. I've removed the "not working" status from your game's entry. :)
Quote from: LimpingFish on Mon 23/06/2008 22:34:09
When people choose to add this kind of game to the database (usually accompanied by "My first ever game! I made it to try out AGS! It's crap, but please play it!") they receive the label "Training Game".
You know, this strikes me as a not such a good idea. If the label "training game" essentially means that "this game is bad", then that would be the exact same thing as giving it a one-cup rating.
Giving a game
both a one-cup label
and a training label is saying it's not only bad, it's so bad that it has to be explicitly stated twice how bad it is - insult to injury, so to speak. And indeed, with a single exception (a game which ought to be in the "short" category anyway), every single training game has one cup. So this extra category doesn't seem to actually help anyone, and it does seem disheartening to whomever gets his games rated like that.
Besides, the more complex the category scheme becomes, the less likely people will actually be able to find what they want. The "game length" tag is meant to, you know, indicate game length (short, moderate, long); not to indicate that "we really think this game is too low quality to be in the database".
Quote from: Radiant on Tue 24/06/2008 00:01:00
Quote from: LimpingFish on Mon 23/06/2008 22:34:09
When people choose to add this kind of game to the database (usually accompanied by "My first ever game! I made it to try out AGS! It's crap, but please play it!") they receive the label "Training Game".
You know, this strikes me as a not such a good idea. If the label "training game" essentially means that "this game is bad", then that would be the exact same thing as giving it a one-cup rating.
Apples and oranges, really.
We could label them Training Games and leave the rating off, I suppose. It wouldn't affect the quality of the game, though, and wouldn't give any indication of said quality either. Training games, in most cases, also happen to be "first game" releases, made simply as an "Hello world!" to the AGS community, and the label would generally be applied in those circumstances.
Joke games, in general, receive 1 cup also. We could leave the ratings off those too, but people would just accuse us of not warning them that the joke wasn't funny. :-\
It would be unfair to rate some games and not others and would pretty much defeat the point of a coherent, rated database of games.
About the added categories. I'm all for a more robust selection of categories in general. Right now there are too few classifications (imo) to satisfy all the games in the database, sports and boardgames being the two most obvious to suffer. I don't see a problem with MAGS being entered as a sub-category because it's an official AGS contest. OROW is not, really, and is run by scotch randomly and when he has the free time and interest. Still, if enough people really wanted an OROW tag I don't think anyone would care, and it's ultimately up to CJ whether more distinctions are added to the list.
And yeah, we will be happy to restore games to working status if the authors update their download links!
Quote from: LimpingFish on Mon 23/06/2008 22:34:09
Quote from: Ultra Magnus on Mon 23/06/2008 04:28:03
Soooo.... the term "training game" basically refers to someone's first game when it isn't any good?
Surely good first games are equally "training", no?
Heartland Deluxe was not the first game I made, but it was the first game I decided to give a proper release to. In learning AGS, I had made numerous "games" beforehand that will never see the light of day. Those are my training games.
When people choose to add this kind of game to the database (usually accompanied by "My first ever game! I made it to try out AGS! It's crap, but please play it!") they receive the label "Training Game".
From what I understand, a lot of people don't release their first game(s), but then a lot do, too.
How about
NES Quest or
Infection? They were both first-ever efforts, and were both pretty darn good.
They're just as much about learning AGS as the crap ones.
Maybe "training games" could just be expanded to "first games" and include the good with the bad.
I'd personally be just as interested in how good a first game can be as how bad. More so, in fact.
While skimming through the new ratings i noticed that 'Principles of Evil I' (http://www.adventuregamestudio.co.uk/games.php?action=detail&id=484&gamesdlrec=%7C38%7C%7C1015%7C&newvoterec=%7C584%7C) was given only 2 cups. This surprised me as it seemed to be a very well recieved game but of course these ratings are subjective so obviously it didn't have the same appeal to whoever rated the game.
What surprised me more though was that such a highly rated game was given such a low rating without a comment from the panel being included. I would be really interested to know why the person who rated this game felt it deserved such a low rating and therefore would like to request a comment be added to the game from the panel.
[EDIT]Actually it would be really great to get comments on any game with a large difference between it's cup and user rating, just so we can see why the reviewers opinion was so different from the communities.
PoE was rated during the first days of work on the new database beginning, when adding comments wasn't a priority.
Notes about the ratings on a number of games were kept, however, so more comments may be added to already rated games in the future.
If you look at the user comments, you'll see a number of issues people had with the game, which may not be accurately reflected in the rating.
Quote from: LimpingFish on Tue 24/06/2008 19:21:36
PoE was rated during the first days of work on the new database beginning, when adding comments wasn't a priority.
Notes about the ratings on a number of games were kept, however, so more comments may be added to already rated games in the future.
Good to hear.
QuoteIf you look at the user comments, you'll see a number of issues people had with the game, which may not be accurately reflected in the rating.
While there are definatly negative parts to many comments, most also paint the game in a positive light (Plus the games thread was largely positive). The general vibe i get for this games seems to me to be "flawed but i enjoyed it non the less". I'm interested to see what the panel thought.
Just in case anyone is interested (and I know bicilotti is!) I've updated my games rating trawl for July 2008 and added in a column for the cup rating which leads to some interesting things:
http://ssh.me.uk/ranking200807.html
The 5-cup games have user ratings of 86-93%
The 4-cup games have user ratings of 67-93% (Lowest: Dance Til You Drop)
The 3-cup games have user ratings of 27-91% (Lowest: Marty Chonks In Dances With Camels, Alien Threat, Ace Quest)
The 2-cup games have user ratings of 29-88% (Highest ranking: Quest for Glory 4 1/2 , Principles of Evil I, Casablanca,The Day After, Trevor Daison in outer space - Chapter one, Maniac Mansion Mania - Episode 9 - radioactive, Maniac Mansion Mania Episode 15: Place Machine, Bernard's Room. Lowest: Adventures of PQT, Don the Dweeb, Henk Stroem in Lost In Cellar)
The 1-cup games have user ratings of 1-89% (Highest ranking ones are Maniac Mansion Mania Episode 2 - Commotion, Rude Awakening!, Unfinished, Battle Warriors : Rovendale Tactics, La Gran Castanya and Rapstar 1.5. Lowest ranking: Proof of Fiction, Scnidersom and Murder)
Quote from: SSH on Wed 16/07/2008 14:41:32
Just in case anyone is interested (and I know bicilotti is!) I've updated my games rating trawl for July 2008 and added in a column for the cup rating
Thanks SSH! That will allow me to put my economectics studies into good (?) use.
Is there a way to include a filter to just show games with translations available to certain languages? It could be VERY usefull, i think. Not everyone have a good enough english to play an entire AG, even when they are able to understand the web page.
Jp
I notice that the script generating the web pages displays n blue cups and 5 - n grey cups. If your browser application window isn't very wide, this sometimes makes the last cup wrap on to the next line, which looks odd. Would displaying a single graphic made up of the five cups rather than five separate graphics fix this, or would it just make all five cups wrap round to the next line if the browser window is not wide enough?