Android and iOS share a common problem: they copied desktop text editing conventions, but without a menu bar or mouse. This forced them to overload the tap gesture with a wide range of actions: placing the cursor, moving it, selecting text, and invoking a pop-up menu. This results in an overly complicated and ambiguous mess-o-taps, leading to a variety of user errors.
It’s less of a problem if you only do short bursts of text in social media or messaging apps. But doing anything more complicated like an email gets tedious. However, in my user study on text editing, I was surprised to find that everyone had significant problems and rather severe workaround for editing text.
With the extremely talented Olivier Bau, together we created a prototype called Eloquent, which offers a much simpler solution. We presented this work at UIST 2021.
This is now one of my favourite articles I’ve ever read. I despise text input and text editing on mobile devices, whether they be Android or iOS. I hate it with the passion of a thousand burning suns, but it seems like nobody else cares. Luckily, the author of this article, Scott Jenson, a man with an impressive career doing UI work at Apple, Google, and others, agrees with me, and together with his colleagues, during his time at Google, he came up with an entirely different, touch-first way of editing text.
The end result – be sure to watch the video to see it in action – immediately clicks for me. I want this. Now. This would be a massive usability improvement, and the fact it isn’t in Android yet, despite being developed at Google, is further evidence Google has no clue how to make good ideas float to the top. Jenson explains why Eloquent, as they called their new input/editing system, won’t ship with Android, while he expresses a bit more optimism Apple might be more open to rethinking mobile text editing:
Unfortunately, shipping something like Eloquent would be challenging. First, as too many people mistakenly see text editing as “done”, there is little appetite to fix it. Second, users have been trained to cope with this error-prone approach for well over a decade. Asking people to change at this point would be hard.
But most importantly, fixing text editing isn’t seen as important enough in the war between Android and iOS. It’s not the flashy feature that shifts your Net Promoter Scores. What I find ironic is that a fundamental change, like fixing text editing, could make people feel much more at ease using their phones and could be an enormous reason to switch. But it would be a slow burn and take years of steady effort. Android just can’t think this way. Apple just might.
Android needs this.
Couldn’t agree more. Text editing on mobile was awful on day one and hasn’t gotten much better. The virtual keyboard is much slower than touch typing but at least strait forward if your patient. However when you actually want to edit existing text (ie delete/cut/copy/paste) it is so difficult because touch screens are so bad at precise input. It becomes trial and error until you happen to select the right range, not to mention the ambiguity of tapping on clickable content when you needed to select it…ugh!!! If I make a mistake or want to rearrange sentences, I often find it easier just to backspace back and re-write the whole thing than to edit it using mobile copy/paste. Despite text editing being just an awful experience on mobile, both apple and google seem to conclude that it’s good enough for them and the haven’t really done anything to fix it.
Honestly I’ve had an easier time editing text using VI on android than editing email text or web forms. Not because VI is designed to be good for mobile mind you, but because VI gives direct control over the cursor via discrete button presses. Moving backwards and forwards is trivial with arrow keys. I think the addition of a universal edit mode on the android keyboard would help, something easy and quick to get to. Buttons that I can rapidly tap or hold down to quickly and precisely navigate to exactly where I want with dedicated copy/paste buttons. Almost anything would be better than what we’ve got today. It really is very bad.
Thanks for the link to this interesting and important work!
While for some reason I always have a very difficult time editing text on iOS on my iPad, on my Android phone I find it to be a bit finicky to position the cursor, but comparatively I find it much easier overall. Maybe it’s just a matter of what you’re used to. I agree Android text editing could be improved as shown in the video, in particular in terms of always showing the cursor handles and allowing drag-to-position, as opposed to only being able to tap to position the cursor. On iOS paradoxically it’s the opposite problem: they already have drag-to-position, but for some reason I always have trouble actually selecting text – it ends up being interpreted as a cursor move. The truth is, the differences between Android and iOS modes of text-editing are great enough that they’re bound to trip up users who switch between the two, as it does for me every time I need to use iOS instead of Android.
This demo looks like it would to some extent make Android text editing more similar to iOS, while still being better than iOS – essentially combining the best of both worlds, and making switching between the two platforms less of an issue. So I wholeheartedly embrace it. The only part I question is the whole “continue dragging to select menu options” part of it – IMHO this change to the meaning of what a drag does, once again could cause confusion due to multi-modality, and could falsely interpret the user’s intention to expand their selection up/down into selecting a menu item. But obviously I’d have to play with it myself in order to make a final verdict.
How many of those problems could be solved by simply adding left and right arrow keys to the touch keyboard?
We will never know…
smashIt,
Agreed. While it’s not going to work generically with normal android apps, you can kind of demo this concept under termux. It adds navigation keys on top of the keyboard. These work just as expected for console applications. There’s no technical reason it wouldn’t work across andorid. After all you can plug in a physical USB keyboard and those arrow keys work everywhere better than precise touch input, but obviously virtual keyboards are too cool for that.
BTW if you want to try this you can only download recent versions of termux under 3rd party app stores because google officially blocks termux-like shells running executables as linux shells do. I worry that termux may be living on borrowed time as all deprecated API levels that support execute functionality could be completely removed from android with no replacement in the future.
Install the Hacker’s Keyboard and find out. 🙂 Adds arrow keys, separate CTRL, ALT, and other keys, and more. Makes working in SSH terminals so much nicer. It doesn’t support swipe typing, so I don’t use it as my primary keyboard, but it definitely makes text editing easier.
The problem is that android and ios never really had a functional text copy and paste function. Just selecting text is more of a hack, windows mobile was way more functional. Yes I said that, it had a functional resistive sensor that allowed you to always select tex, even from dialogues. Almost everything (text) was selectable like the windows counter part. While google and apple segregated features from the desktop and even made it a dirty word. Now we have mobile oses that don’t follow normal conventional usability trying to hack back desktop features (keyboard on Ipad???). While Google keyboard and Hackers Keyboard (legacy) do solve some problems. Android and IOS still treat text as a special thing, and copy and paste
missingxtension,
Windows mobile was a decent mobile platform…but it never had critical mass to compete against the dominance of IOS and Android. Same with webOS. I really don’t think it’s the lack of merit that is sinking competitors, but the harsh reality of network effects. I doesn’t matter how good a platform is if it doesn’t have users and developers to lift it. Apple and google are very successful at keeping customers on a short leash making life very challenging for newcomers and customers who want to jump ship.
I genuinely wish mobile could have ended up with more competition, and yes that includes microsoft. This market is in desperate need of more choice and innovation. But on the other hand I’m slowly learning that my utopian vision of a healthy market with competition and meaningful choices for everyone might have always been a fantasy. I know that given the opportunity, microsoft would have gleefully locked down the market to competitors just like apple and google have done. This is the problem with market winners, they all seem to share the same destructive tendencies: slamming doors shut behind them so that no more competition gets to enter the ring. The is what’s wrong with all our markets; there’s no balance whatsoever. The field becomes so asymmetrically biased towards dominant corporations that those at the top can just take their position for granted even when they stop innovating and do little to continue earning their huge share of the wealth. Meanwhile those who are most eager to innovate are left fighting for such small portions of the pie that their reality is a subsistence economy at best.
https://www.osnews.com/story/137210/nearly-500-brands-exited-smartphone-market-during-2017-2023/
Some intriguing ideas there. I like what I see!