Effective GUI And Interaction models for touchscreen

[benco]

Hi,

It's been a while since I wrote a message on here... I'd like to discuss one aspect of adventure games.

I'm furiously thinking about making adventures games on tablet and mobile phones since several weeks but I'm facing big questions about the way to interact with the touchscreen. The traditional "point 'n click" GUI model is not applicable here. So I'm looking for some informations, design, reviews, testimonials and stats... in order to optimize the user experience for my "future" adventure game on these kinds of platform.

To be honest, I didn't take the time from now to buy and/or try some games on touch screen (for instance, buying Monkey Island I & II on iOS or installing scummvm on Android - maybe I should do this to make all clearer). I just watched videos about people recording their game sessions ...

What I note as typical interaction models are:

• single touch on the screen (the most evident one of course )
• touch and drag on the screen with the same finger (move the cursor positioning just above the player's finger position)
• touch and drag on the screen with two fingers (move the cursor positioning just above the first finger position with your other hand)
• touch the screen long enough to enable "pixel perfect touch" (a zoom window appears above the player's finger position)
• touch and drag on the screen with the same finger with automatic zoom window
• other ones... ?

What do you think about these solutions ?
What are the most efficient and/or convenient ones according to you ?
Are they really the best way to play adventure games on touchscreen ?

Thanks in advance for your help and your contribution !

Best,

Benco

[KBlaney]

Human interface guidelines for the older iPads suggested that buttons be no less than 45 pixels square. I think a lot of what Vince Twelve said about inventory items applies to your finger as well, that is, the exact touch point is going to be vague and so making the point of interaction as friendly as possible is advisable.

If you haven't played Machinarium, that has a pretty good interface on tablets.

[benco]

Thanks for your reply.

Indeed, "pixel perfect click" is not achievable anymore on touchscreens without "additional tools". "Tooltip" informations aren't available anymore too... (like in the following example)



So, the way we must interact with the tablet could potentially modify the logic and the gameplay... That's why I looking for some good advices before starting the initial work...

Thanks also for your suggestions. I played the flash version of Machinarium a long time ago... I'll give a while to the iPad version 

[KBlaney]

I made a framework in flash for making a game like Myst during a gamejam. One of the issues I ran into was being able to see what was clickable or not (no tooltips, and I couldn't change the pointer icon like Myst did). My solution at the time was to add a GUI button that highlights all clickable things on the screen for about a second. I'm rather new to AGS, but I suppose having clickable objects animate (suppose they display their name) for a second in response to a GUI button could work.

[Vince Twelve]

I think to make a really good point-and-click-esque adventure on touch screens, you have to do more than just straight port the concepts from mouse-driven p&c's to the touch device.  The interface must be radically redesigned.  (This isn't always possible, when you're porting a game of course. But if designing a game from the ground up with touch in mind.)

Touch devices lose these abilities:
-No tool tips or mouse-over feedback
-No mouse-position-triggered guis (i.e. an inventory that slides down when the mouse is near the top of the screen)
-No right-click
-Keyboard not as easily accessible
-Less precise clicking ability

But gain these:
-Drag and drop (touch and drag) easier and more natural
-Swiping the screen, dragging around, and moving your finger from one side of the screen to another much easier and faster than doing the same with a mouse
-Multi-touch ability
-Gesture recognition

So you need to design the interface to avoid the things you lose and take advantage of the things you gain.

One idea I've thought of is, instead of touching the ground to walk the character around, you could drag the background. So, on a long scrolling hallway, instead of clicking where you wanted the player to walk, waiting for them to walk there, and then clicking again after the screen had scrolled over a bit, it would be faster for you to just take your finger and drag the whole background to the side in the same way that you would scroll a website or email, but horizontally.  The player character would then walk or run (or warp if they fall behind) to keep up with your scrolling.  This would be a hassle on a mouse-driven PC game (imagine having to click and drag the background horizontally several times to move from one end of a hallway to the next.  Annoying).  But on a touch device with scrolling momentum, using your finger to flick the background to the side a few times to cross a long hallway would be quite easy and much faster than clicking and waiting for the character to walk.  You could also implement a replacement for tooltips this way, by highlighting any interactable items near the center of the viewport for example.

The "best" solution will be the one that fits your game design.  This idea doesn't work for games that aren't composed of long scrolling hallways, for example.

But that's just one idea.  You've got a whole new set of possibilities, think outside the mouse-driven box.

[benco]

Thanks for your reply, Vince.

You've got a whole new set of possibilities, think outside the mouse-driven box.

Yep, that's what I'm trying to do ... I was just wondering what was, according to players' experience, the most convenient/efficient way to present the GUI and its interaction model.

I think I'll try some games on my iPad to analyze it further...

[Khris]

I like what TeamViewer does: the touchscreen is handled like a laptop's touchpad, so there's a visible cursor on the screen, and dragging makes it move accordingly. That way you can keep mouseover stuff. I believe ScummVM uses this, too.
Since the location of the tap is irrelevant in this approach, you can split the touchscreen into areas; say tapping the left half is a left-click, and tapping the right half is a right-click.
Dragging near the edge produces scroll wheel input, etc.

This completely eliminates the usual awkwardness of touchscreen input for mouse-driven games.

It's also a pain in the ass for people who dislike touchpads though.