Linked by Thom Holwerda on Thu 27th May 2010 22:47 UTC
Graphics, User Interfaces Now this is an interesting article, written by Lukas Mathis. He makes the interesting case that the gesture-based interface, as seen on the iPad and many modern smartphones, is actually more akin to the command line interface than the graphical one. He also offers a number of solutions for that pesky problem of gestures being anything but discoverable.
Order by: Score:
by WorknMan on Thu 27th May 2010 23:40 UTC
Member since:

He argues that this non-obviousness inherent to current gesture-based interfaces means they once again centre around memorisation instead of recognition.

Ha, not hardly. Though certainly not perfect, something like an iPad is still a hell of a lot more intuitive than a CLI, assuming you're approaching both for the very first time. I mean, don't get me wrong... the command-line does have its strengths, but 'discover-ability' is not one of them.

I will agree that a gesture-based interface isn't going to replace a keyboard and mouse for getting real work done anytime soon, but they're sure handy to have around when you're kicked back on the couch and want to browse the web, or whatever ;)

Edited 2010-05-27 23:43 UTC

Reply Score: 2

RE: Memorization
by joshv on Thu 27th May 2010 23:58 UTC in reply to "Memorization"
joshv Member since:

"help" typed at any command line is reasonably good at directing you in the right direction. Perhaps we need a question-mark gesture that displays a list of gestures with directions.

Reply Score: 10

RE[2]: Memorization
by vodoomoth on Fri 28th May 2010 11:48 UTC in reply to "RE: Memorization"
vodoomoth Member since:

Great idea! and there's no irony involved.

You should have patented the gesture before telling us about it. I'm sure it would have been granted.

Reply Score: 2

RE[3]: Memorization
by sbenitezb on Fri 28th May 2010 13:04 UTC in reply to "RE[2]: Memorization"
sbenitezb Member since:

probably the concept of gesture is already patented.

Reply Score: 3

RE: Memorization
by galvanash on Fri 28th May 2010 00:47 UTC in reply to "Memorization"
galvanash Member since:

Everything you say here is true, but none of it invalidates the point of the article, namely that gesture based UIs require memorization instead of recognition.

That said, nothing in the linked article is particularly insightful either - this is a very well known issue and predates the multi-touch interface as Apple implements it by many years...

Apple did not fix this problem - it is rather inherent to the interface paradigm. What they did manage to do, however, was promote application interfaces which by design are limited to 3 obvious and simple control behaviors, i.e. touch to click, drag to scroll, and pinch to zoom. And they mostly stick with these.

Most applications do not attempt to go beyond these 3 mechanisms for their primary control mechanics. Some implement extended gestures, but usually only, as the article suggests, for shortcuts that are not required for basic interaction with the application.

As the applications get more complex the inherent problems with gesture based interfaces become more obvious. Another post here mentioned the possibility of a "help" gesture. This is probably a very good idea for those applications with UI paradigms which simply cannot cope with being limited to the 3 basic gestures.

I would also suggest that such applications are trying to fit a square peg in a round hole so to speak. What I mean by that is some of them will certainly become popular for what they can do as a tool, but it will be in spite of their bad UI, not because of it. The more popular applications in the appstore tend to stick to the basics and simply come up with new ways to utilize them.

Reply Score: 3

RE[2]: Memorization
by WorknMan on Fri 28th May 2010 05:31 UTC in reply to "RE: Memorization"
WorknMan Member since:

What they did manage to do, however, was promote application interfaces which by design are limited to 3 obvious and simple control behaviors, i.e. touch to click, drag to scroll, and pinch to zoom. And they mostly stick with these.

Well, and you also hold down on stuff, like moving things around the home screen, and copy/pasting text. So technically, that's 4 ;) And to me, that seems ok, given the limited scope of the touch interfaces. One thing I really miss though is tooltips. Sometimes, I wish I could just hover my finger over a button or some other icon to see what it does. No such luck though ;)

Reply Score: 2

RE: Memorization
by sultanqasim on Fri 28th May 2010 00:52 UTC in reply to "Memorization"
sultanqasim Member since:

Ha, not hardly. Though certainly not perfect, something like an iPad is still a hell of a lot more intuitive than a CLI, assuming you're approaching both for the very first time.

That's because it uses a mix of GUI and gestures. The GUI part of the UI is very easy to understand.

Still gestures are much easier than command lines. No exact spellings of commands to remember, just a basic idea on the gesture is usually all you need to know. No command syntax to memorize either. man pages are very helpful for that though.

Reply Score: 2

RE[2]: Memorization
by Gone fishing on Fri 28th May 2010 05:30 UTC in reply to "RE: Memorization"
Gone fishing Member since:

Nice article, yes gestures are simpler than the command-line but also much less powerful. When comparing power to simplicity ratio they would look similar.

Exact spellings in the command-line make it difficult - true - but we also have tab auto complete and the ability to scroll through recent commands.

I'm beginning to have the opinion that the increase of features in IT devices to make the easier to use, in fact, often makes them more complex and difficult to use.

Oh damn I sounding like a power of the command-line. bore

Reply Score: 5

Makes Sense
by th3rmite on Fri 28th May 2010 00:23 UTC
Member since:

At first when I read the summary, I was thinking there is no way in hell you can compare gestures with the CLI, but then I actually read the article and the author has a valid point. Gestures are non-obvious and force the user to rote memorize how to get things done unlike the graphical menu system.

Reply Score: 3

RE: Makes Sense
by Karitku on Fri 28th May 2010 09:36 UTC in reply to "Makes Sense"
Karitku Member since:

At first when I read the summary, I was thinking there is no way in hell you can compare gestures with the CLI, but then I actually read the article and the author has a valid point. Gestures are non-obvious and force the user to rote memorize how to get things done unlike the graphical menu system.

Same can be said on most GUI programs. Almost all people are educated on usage of programs and all work in GUI enviroment is also largely based on memorizing different things. Same goes gestures however they can be much more simple (and they are) and they can remind something we used to do so they are easier to learn.

Whole point of NUI research is to transfer something we used to computer enviroment as similiar as possible yet making it more efficient. Reason why NUI is better than GUI is because it has all same powers as GUI and yet have power of gestures which we have used on something else and already know them. One great example is usage of colors in GUI to explain different things, even baby reacts differently on different colors, however in CLI world you need to first either understand commands or understand language to accomplish same.

Reply Score: 2

RE[2]: Makes Sense
by vodoomoth on Fri 28th May 2010 12:18 UTC in reply to "RE: Makes Sense"
vodoomoth Member since:

Whole point of NUI research is to transfer something ...

What is "NUI"?

Reply Score: 1

RE[3]: Makes Sense
by Zifre on Sat 29th May 2010 12:39 UTC in reply to "RE[2]: Makes Sense"
Zifre Member since:

What is "NUI"?

Natural User Interface.

Reply Score: 2

Wrong conclusion
by Ford Prefect on Fri 28th May 2010 10:06 UTC
Ford Prefect
Member since:

From the article:

I also happen to believe that despite the solutions Mathis provides us with, this lack of discoverability will ensure that gesture-based interfaces will always play second fiddle when it comes to getting real work done.

Sorry Thom, but this is the wrong conclusion. The relationship between CLI and gestures is an interesting observation. But neither of them are, just because of discoverability issues, hold you back from being productive.

The CLI is used since decades to get real work done. The same goes for Photoshop, 3DS Max, and any other software targeted at professionals, including the "creative" sort.

I would even claim that memorization is the key to productivity. In fact, in behavioral psychology you see a lot of work related to how the human brain adapts to frequent tasks. It all boils down to
a) improving the memory structure (e.g. to memorize a chess board, learn to cluster it into a smaller set of entities)
b) gaining a HUGE memory (e.g. of common and then less common chess constellations)
c) in this memory, not only learn states for quicker reaction, but also the needed reaction itself

So if you work long enough in Photoshop, you don't search for that item or menu entry anymore, you don't even ask yourself which tool you will need next. In the CLI, you don't think about the commands you type, and sometimes, you won't even think about typing a command at all.

The same goes for gestures and any other interface that you may master with memorization. Shortcuts/Key Accelerators are so popular on the Mac platform for the very same reason.

Reply Score: 4

consumer electronics
by l3v1 on Fri 28th May 2010 11:35 UTC
Member since:

As great as the iPad and Android tablets may be, they're made for consuming, not for creating.

True. If you look at them from this point of view - targeted use cases aiming for consuming content - then the whole issue is not such a big deal. It's not a big problem that productivity-wise (let alone development) gestures are worse than traditional interfaces, because they are not - at least I sicnerely hope - intended for productivity and development tasks. And you don't need a gazillion gestures for a usual consumer product, just a bare minimum that enables the content consumption, and those can be memorized fairly quickly.

Discovering them is another issue, consistency between devices - regarding gestures - is another, but they all can be mastered given enough practice, if the divice - or the content accessible through it - is worth the effort, that is.

Reply Score: 2

Comment by LighthouseJ
by LighthouseJ on Fri 28th May 2010 12:40 UTC
Member since:

I think we've all debunked the topic sufficiently, but gesturing still has a very long way to go be useful.
From trying to write with my Palm Pilot iii years ago to now with some UI interfaces now that offer gesturing, it's an incredibly inefficient process to perform a simple task. It's a giant Rube Goldberg machine.

It's not until multi-touch interfaces where more than one input per instant seems to make the process easier to use. Once you have that, make it intuitive for humans (let humans, not programmers, decide how to use it).

Reply Score: 1

RE: Comment by - Graffiti
by jabbotts on Fri 28th May 2010 19:00 UTC in reply to "Comment by LighthouseJ"
jabbotts Member since:

I miss Palm's original Graffiti alphabet; high speed input, low rate of error in recognition.. fantastic. I've yet to see a alphabetic gesture input as accurate outside of an onscreen keyboard (lower speed, low rate of error in recognition).

Reply Score: 2

Member since:

I think there is a fundamental difference in memorization a command and typing it, and using gestures. I currently use a MacBook with jitouch ( which offers a vast number of gestures. Even for things like closing a browser tab and filling out my saved passwords on a website, I have a gesture.
At last for me, those are very different than the commands I on the CLI. For remembering the CLI commands, I have to use brain/interlect whereas for the gestures, it is plain instinct (after some training of course). I think that I want to close a tab, and my hand does it. No usage of my brain needed. Just the "brain" of my hand. Now when I use the CLI, I still have to activly "think" and type. This goes really fast most of the time, but still it requires much more "brain usage" than gesture.

I hope I could make my point clear, I am no native english speaker ;)

Reply Score: 1

darknexus Member since:

Actually, at the core, they're the same. I do a lot with the cli, and I don't even think about what commands I need most of the time. I need a list of files, I type ls before I even realize what I've done. It's memory, in both cases. You think you want to do something and immediately your gesture comes to mind, I think of something and the command is typed before I even think about it. As far as the brain goes, there's no difference except *what* is being remembered. Gestures and the cli are certainly not the same interfaces, but the memorization behind them is identical.

Reply Score: 2

TTy23 Member since:

That the memorization is not the same is exacty what I wanted to say. The CLI stuff is "stored" in the symbol store of the brain, whereas the gesture is stored in another part of the brain (where movement is handled). And I think that the "movement store" is easier to access than the "symbol store".


Or to put it more logically: Typing a command on the CLI involves a lot of gestures, since you need to type the keys (since each keypress is a gesture), and a gesture is just one.
So in the end you have:
memorization, gesture, gesture, gesture .....


If one assumes each gesture involves the same memorization as a CLI command it is:
memorization (command), memorization(key), gesture(key), memorization(key), gesture(key), ....
memorization(gesture), gesture

Of course only if you trained the gesture as much as typing. Which probably most people will do once there is a set of standard-gestures.
So I would assume a gesture is more similar to a keypress than to a CLI command.

Edited 2010-05-28 19:18 UTC

Reply Score: 1

jabbotts Member since:

"after a bit of learning of course" is no different for command line. Gestures used regularity become second nature "instinct" as do commands typed regularly. I spend much of my cli time in *nix and often get an error or two from "ls *.*" or similar when I first switch over to a Windows cli. Previous to my *nix days, the Dos and Windows cli commands where pretty much automatic; think what I want to do and the commands are on the line before I realize I hit enter.

Reply Score: 2

Not only gestures, voice control also
by rimzi on Fri 28th May 2010 15:41 UTC
Member since:

Not only gesture-controlled UIs, but also voice control seems to require much more focus and thinking.
If you ever tried Vista/7 voice recognition, you might acknowledge that.

Reply Score: 1

To be honest
by deathshadow on Sat 29th May 2010 07:14 UTC
Member since:

I hate gesture interfaces and touchscreens - always have. Admittedly, much of my dislike for touchscreens comes from the fact I can't stand fingerprints on my screens ;) but moreso I find them really counterintuitive compared to a normal pointing device, especially when half the time I can't see what my ham-sized finger is actually OVER. (same reason I can't use Saitek joysticks, they feel like they're sized for five year olds or girl-hands - certainly not my oven-mitts) Using a stylus alleviates the problem a bit, right up until you lose the bloody thing.

Though the REAL thing I hate is the complete lack of tactile feedback. I love the good solid "SNAP" of a switch registering (why I hate dome/membrane keyboards) - There's a reason I'll probably be using a Model M keyboard on my workstation until the day I die. (I've got enough of them stockpiled it's no joke).

Admittedly for small devices they make a lot of sense since you can't slap a giant keyboard or mouse on one - but that's a sacrifice that would leave me looking to try and come up with some other solution BESIDES the touchscreen... like say a D-Pad and a couple small buttons.

Edited 2010-05-29 07:16 UTC

Reply Score: 2