AGS review system

[Krazy]

Well, welcome at last Jozef and thanks for being the only guy ever to review a Princess Marian game 

And Timothy Lande 

[Jozef]

Speaking of Timothy Lande, just wanted to give you a short update on the project I was working on with Archive.org.  We wanted to create a database of free games there, and I was responsible for the adventure section.  Unfortunately, the project was not as sexy as publishing TV shows and music, so it was canned for now.  Maybe one day it gets revived, when there's nothing else to publish.

[AGA]

Heh, good to see you come out of the shadows, Jozef. Your column always comes in handy for choosing which stuff we (AG Underground) will cover in more detail.

[Jozef]

Oh well, I decided that it was time to step out of the closet

And thanks for the really generous coverage of my column over at AGU; I really appreciate it.

[Scummbuddy]

Jozef, did you ever get to reading the comment I left on the March 2005 edition of your article. netmonkey linked me to your site, which I appreciated your words on my game, but I left a comment to hopefully straighten somethings out.

[Pelican]

I'm not sure if someone could implement this. I think that the point system should be governed by moderators who would read through posts in technical forums, critics lounge and completed games page, when something useful or helpful has been posted they would award these points, or we could have where any user can give karma points, but restrict it to 1 point per post so that people don;t start spamming.

Actually, I think the Sims 2 bulletin board does something like this. If someone gives you a really helpful reply, you can give them a 'bene' (rate this post beneficial). They all seem quite obsessed with collecting lots of these, though I'm not sure why. Perhaps they get some sort of bonus out of it? Or 'cred'? It could work here. Dunno what you'd do for incentives though...

[Jozef]

Jozef, did you ever get to reading the comment I left on the March 2005 edition of your article. netmonkey linked me to your site, which I appreciated your words on my game, but I left a comment to hopefully straighten somethings out.

Yes, I did see the comment, but at that time I've been still working on my Master's degree, and didn't have the time to check the site every day.  So by the time I saw it, it was about two weeks old.  Still, I think you did a very good job explaining the background to the game; in fact, I'm always happy when the authors provide more information.  After all, they know more about their games than I could ever learn.

[Ponch]

Hi there, Jozef. Thanks for de-lurking and thanks for all the work you do at DIY Games. Most of the games I download were the ones you rated highly.

Thanks also for the great reviews you game the Barn Runner games. They made my day!

- Ponch

[Rui 'Trovatore' Pires]

While we're at it, let me tell you what oldgames.nu does. That's a pretty good and comprehensive abandonware site, and you have some "points" and "download credits" you start of with. As you download stuff, you use up your credits. To get more, you either participy in the forums (and they take care not to allow spam, and believe it or not I did not find much of that there, and what I found was quickly crushed) or send a review/info/faq/cheats/whatever. It is rated accordingly, if it's a review it's rated in a 0-20 scale. Well, it worked for me - my first review got a rating of 4, and I thought this wasn't enough, so I spent some time on another review. That one earned a 9, so I pulled myself together and started really reviewing, and from then on I got straight 20's all the way (except a coupla times when I had 19). It motivated me. I don't know how this could be adapted to here, but I thought I might as well share.

[magintz]

How this could work is that every member is given five 'spending points' that they can use to give to another member. Once they have used up the five points the only way they can get more is by reviewing things, writing critiques etc...

A good way to have this would be to begin with a trial system maybe, where we have a forum post in the General Forum. About three or four people could keep track of it, every time someone posts something useful one of the 'moderators' would begin by adding that person a point. Should other forum members wish to give points they can merely reply in a follow-up post.

I think this would be a great way to test whether a system like this would work, and if so we can then put it to CJ to implement.

[Snarky]

Hmmm... I don't think that would work very well. Studies show that online communities have very unequal levels of contribution. Typically, you'll find that 80% of people who read a forum just lurk, while only 20% post. Of the people who participate, the 10% most active posters usually contribute more than 50% of the posts. (This phenomenon is called the Pareto principle.) The most active posters (generally) add the most value to the community, but you need the less active posters, too, because they provide an audience that motivates the posters to post, and a pool to recruit from. (In the AGS Forums case, people primarily add value to the community by creating games, making art to be critiqued, offering coding help and constructive criticism, and -- if we make a review system -- by reviewing games. Just posting adds some value, too, if it's a good post, but not as much.)

Therefore, it's very important to not create barriers that make it difficult for someone to contribute at a low level. Some people may just like to contribute ratings, without writing reviews themselves. If they can't do that, you'll drive them away, and they'll never get to the point where they want to write their own reviews. I would think the only reason it works for the abandonware site is that people need the points in order to download things. That's an extrinsic motivation. However, to get people to participate in the AGS Forums, we need to motivate them in intrinsic ways.

Test it, by all means, but I would be very cautious about implementing something like this.

[Kinoko]

I still think it's just an awful idea to try and make people do reviews. People who want to do it will do it, and those who don't will likely never want to do them. Economics, people! ^_^ Let people do what they want, otherwise you'll scare them away. I'm one of the people who would never, ever, ever do a review unless I suddenly got this amazing urge to do so. No amount of 'incentive' would do it for me.

I do think of this like a market place. Let the consumers do what they want. If you make a game and people want to review it, whoop-de-hey! Otherwise, you just have to be satisfied with whatever feedback you get. There are already HEAPS of places on the net that review just about everything that comes out (I actually got sick of reading all the Cirque reviews, it started to feel like a chore) and if they don't, you can email them and suggest your game, or just accept that noone wants to review your game. Maybe it's just too crappy, who knows? Besides, reviews are nice but they aren't the be-all and end-all. If you don't have many reviews, it doesn't mean your game is bad (it COULD, but not necessarilly).

The best thing you can do is make a game so good it just sells itself. There's nothing worse than knowing you had to shove it in people's faces. I'm getting a -tad- off-topic here but I think it's all related

[Snarky]

I have to disagree with you there, Kinoko. Yeah, some people will write reviews even if it's not encouraged, and many people won't write reviews no matter what. However, I think there's a significant number of people who would like to contribute reviews, if the community signalled that it was a valued contribution. Sometimes that can be as simple as making a space for it. Sometimes a level of recognition will do the trick (a category in the AGS Awards?). Sometimes a role-model can inspire people.

We don't have to be satisfied with the way things are. Changes we make to the system, initiatives we start... these things can make a difference to how others act. People do respond to incentives.

You may not ever write a review. Well, I may not ever compose any music, but that doesn't mean we shouldn't encourage people to. We have the music competition and a category in the AGS Awards. I'd bet that has increased the amount of music produced in the community, and probably raised the quality of it, too.

I do agree that we can't force people to do something they don't want to. We can only encourage, and offer incentives. Fortunately, I think that's sufficient.

[Kinoko]

True, true. I agree, a proper space for reviews could be well received. The major problem I have is when people feel that they have to write reviews in order to receive basic benefits. I think the points system is terrible, because that's like forcing people to do reviews. Downloads should be freely available. I don't want people to 'earn' my game.

[Ponch]

I agree with Kinoko. I dislike the idea of points and can't imagine a scenario where I would ever review a game. I'm terrible at those sorts of things. My review would be something like:

"I liked it only quite a bit. The end."

If the AGS community wants a user review system that's better than the user comments on the individual game pages, then that's cool. Everyone who wants to review will and those of us that don't write reviews don't lose anything due to our lack or incentive or weak natural reviewing abilitites (there may be a genetic link, you know).

I mean I like Two of a Kind and Cirque De Zale. They were probably my favorite games of last year (after Barn Runner, of course  ) but I don't think I ever posted anything in their threads to that effect... or anywhere on these boards for that matter.

I'm all for user reviews and everything should be done to facilitate them if that's what people want. I just doubt I'll contribute, that's all. More power to those that do.

- Ponch

[Snarky]

The major problem I have is when people feel that they have to write reviews in order to receive basic benefits. I think the points system is terrible, because that's like forcing people to do reviews.

I agree completely. It wouldn't be very friendly at all. Apart from that, I also don't think it would help build the community.

Downloads should be freely available. I don't want people to 'earn' my game.

Yes. I don't think anyone meant to suggest that you would have to earn points to download (apparently the points would only be required to rate other people's reviews), but what you describe would be a terrible idea.

One reason why I generally like the idea of encouraging reviews is that I would like to read them. Another is that I can realistically see myself writing one. Actually ever completing a full game, on the other hand... However, since this forum is supposed to be primarily for game makers (or at least wannabe game makers), adding features to attract fans and critics (who are not working on or intend to work on a game) may not be the way to go.

[Rui 'Trovatore' Pires]

Ok, I was just saying how it worked in that site, I didn't really expect it to be ported over here. After all, like someone already said, it IS a case of "you contribute or else you don't download". That's fine in a site that's all about downloads... and this ain't the case.

[SSH]

Originally when I started the FOREGOs the idea was to encourage people to contribute well to the forums (sexiest AGSer encourages people to post hot pics of themselves, woo!  ), but the problem is that unless you're in the top 5 or so for a year then you dont get any recognition.

So the idea was to encourage people to do stuff. I never wanted a system that would discourage or prohibit anyone from doing anything!

The problem with reviews, is that unlike everything else creative to do with AGS, they don't actually come as part of the forum. Completed Game threads tend to get short comments, rather than reviews. There needs to be a way for people to say "Great game" and ways for people to give an in-depth review, and for those looking at a game to find the reviews. Also, people with a general community interest migth like to see all the most recent reviews. I dont think the reviews on the games page are really moderated, which would be required to stop malicious or spurious reviews getting in the system. So how about having some more Games page moderators nominated (how many ar there now?) and woudl it be possible to have a "Game Reviews" forum that cant be posted to directly, but rather any review posted on a game's page gets mirrored as a forum post?

Does that make sense or sound silly?

[Rui 'Trovatore' Pires]

It makes sense, I second it.

[Jozef]

I'd like to point out the forum for the Maniac Mansion Mania games (sorry; it's in German).Ã,  Whenever somebody posts a new game thread, it's in the form of a poll, with a five-pint rating scale.Ã,  This encourages the forum goers to rate the game, and to write short reviews.

I haven't been active here for long enough to see if it worked, but if it did, the voting could be time limited, and once it's closed the overall score would be posted in a sticky thread, which would capture all the games, with a link to the original thread for people to add their reviews or read others'.