Self-driving Cars: User Interface Will Be The Key To Success

Volvo recently conducted a survey and asked consumers about their perceptions of self-driving cars. The question that stood out to me was whether a car company like Volvo or a technology company (Google, unnamed) was best positioned to bring safe self-driving cars to the market. Volvo was obviously fishing for a particular answer, and while they certainly have a vaunted reputation for technical innovation in the service of safety, I’m afraid I can’t go along with the answer they’re hoping for, partially because safety is only part of the story. In my opinion, no car company working alone is going to be able to produce a self-driving car with the kind of usability that consumers will expect. And for self-driving cars, usability is just as important as safety. In fact, they’re inseparable.

I had the pleasure of visiting Volvo’s factory in Sweden last summer and driving their new flagship car, the 2016 XC90, around Sweden and Norway. The new Volvo has an array of sensors and controllers that give it the vision and situational awareness that self-driving cars need. Its cruise control monitors the speed of the car ahead so it can moderate its speed to match. It will brake and even panic-stop to avoid a collision. It can identify cars and other obstacles in 360 degrees and warn you if you get too close. It watches the lane markers so it can nudge you back into your lane if you stray. If it were programmed to do so, it could easily auto-piot on the freeway. But unlike Tesla, who enabled autopilot software and saw its owners immediately engage in risky behavior, Volvo’s autonomous features are strictly for countering inattentive driving, rather than encouraging it. As a self-driving car enthusiast, I personally would prefer the Tesla approach, as I feel that the Volvo’s features are a waste of potential. As a rational person, however, I think they’re probably wise to be prudent.

But even if Volvo or Tesla or BMW or any other company that’s flirting with autonomous operation were to put out a car that could see the road and the other cars and maneuver itself safely from point A to point B, that’s only scratching the surface of the self-driving car future that techies like me are anticipating. Why don’t I think that a car company will be able to bring that future to pass by itself? Allow me to introduce Exhibit A: the built-in navigation system.

The navigation system in the 2016 Volvo XC90 is of above-average quality compared to nav systems in other high-end cars. That is to say, it’s terrible. Now, once you’ve entered in the address of the place you want to go, it’s pretty good. In fact, one of the coolest things about the Volvo is the fact that it has a huge, well-positioned display in the dash, and it even displays a mini-map in the instrument cluster, between the speedo and tach. Higher-spec versions even show turn data in a heads-up display. But the Volvo’s weakness is shared among all built-in nav systems in every car I’ve experienced. It’s time-consuming and unintuitive to tell the car where you want to go. If you have a complete address, it’s just slow. But if, heaven forbid, you need to actually search for something, it’s just awful. And if you’re actually driving when you’re trying to hunt and peck, you’re risking your life. The Volvo has a touchscreen that supposedly lets you draw the letters of the place you want to go, but it’s worse than a 1990s era Palm Pilot, and it also has a voice recognition function that’s substantially worse than Gen 1 Siri.

Now, I mention navigation systems not only to illustrate that even in expensive cars, car companies don’t have a good track record of deploying intuitive technology, but also because the navigation system is arguably one of the cornerstones of a truly usable automated car. Having a car that can drive itself isn’t very useful if it’s hard to tell the car where to go.

Sure, if you’re already on a long freeway, it would be nice to have a car that could safely drive forward until it ran out of gas, but what we all really want is a car that can drop us off in front of a busy concert venue, then pick us up when the show’s over, and we don’t really have to worry about what it’s doing while we’re in the show; a car that can take one of our kids to ballet lessons while we’re at a doctor’s appointment with the other one. To truly unlock the potential of the self-driving car, we’re looking toward a future in which owning a car becomes an unnecessary luxury, since most people will find it more convenient and cheaper to subscribe to a service that brings a car to your doorstep on demand, takes you where you need to go, drops you off, drives away, and you may never use the same car twice. For this future to be realized in any reasonable way, cars need to become very good at understanding where you want to go and what’s really the best way to get there.

When Apple announced in 2012 that it was releasing its own maps app for iOS, I was pretty excited. The mockups it showed were very cool looking. The first time I tried using Apple Maps in an unfamiliar city, it took me to a residential street in Palo Alto that was a considerable distance from the office of the VC firm whose address I had entered into the App. It didn’t take me long to realize just how much I’d taken Google Maps for granted. Recently I was traveling in Italy, using the Navigon app to get around, which is usually pretty reliable. Apparently, Navigon’s data on the Aosta valley is not very good. It tried to send me down dirt footpaths. It asked me to go on one-way streets the wrong way. It sent me into restricted traffic areas with no warning. Worst of all, it kept trying to coax me off of major bypass roads and into narrow urban downtown streets, presumably because that was a more direct route. Imagine either the Apple Maps or Navigon scenarios playing themselves out over and over with self-driving cars and you can start to imagine how high the stakes are.

But let’s assume that any self-driving car is going to at least navigate at a Google Maps or Waze level, and be able to find the correct location and a good route, aware of local traffic conditions. It won’t be enough. People don’t always drive in a strictly point A to point B fashion. Sometimes they need to search for places and they don’t know the exact location. Google maps doesn’t have a “drive me around this general vicinity and I’ll know it when I see it” feature.

Let’s leave aside self-driving cars for a moment and talk about where navigation systems for old-fashioned manual cars need to go. What I’d like to see in the next Android or iOS update:

“OK Google: take me to Ikea.”
“Do you want to go to the Ikea in Springfield or the Ikea in Shelbyville? With traffic the Ikea in Shelbyville is four minutes closer.”
“Shelbyville.”
“OK, navigating to Ikea. One Ikea Way, Shelbyville.”
— Five minutes pass —
“OK Google: we need gas.”
“The least expensive gas on our route is at the Chevron at 1123 Route 28, shall I navigate us there.”
“Yes.”
— Five minutes pass —
“OK Google: I need Starbucks.”
“The most convenient Starbucks on our route is right behind us, one minute away. Shall I navigate us there?”
“Yeah, Google, when I saw us pass it I got a hankering.”
“OK, navigating to Starbucks.”

For version 2, Google would actually be telling us we need gas instead of the other way around. (It will probably be a while before Google knows we need coffee before we know it ourselves).

There’s not much difference in that scenario between a manual car and a self-driving car. The need for high-quality information about the places in your area and how to get to them is similar. The need for a leap forward in natural language computer interfaces is also similar. A lot of people are working on these kinds of intelligent, conversational voice-based user interfaces, and self-driving cars will only be a small part of their eventual impact on computing.

Now, the most important quality of a self-driving car is going to be the avoidance of catastrophe. During the scenario that I just outlined, I completely ignored whether a small child bounds into the street in front of me, or whether a low-lying roadway is flooded, or lumber from a truck falls off and goes skittering toward my windshield at highway speed. Nevertheless, some of the most difficult engineering challenges are going to be about identifying suddenly-appearing danger and maneuvering the car appropriately. Plenty of ink has been spilled in the press about whether our self-driving cars will have to be programmed to decide to kill either the car’s occupants or pedestrians if catastrophe is unavoidable. Though I don’t give much credence to dwelling on the nightmare scenario, considering how much safer even the most rudimentary self-driving car technology is going to be, I’m also confident that car company engineers are going to do a pretty good job of making their cars react to dangers.

I’m less confident in their ability to avoid non-lethal catastrophes. Consumers will not have a lot of patience for cars that take their occupants to the wrong location, attempt to drive on footpaths, try to board non-existent ferries, or get stuck at intersections by being overly cautious in response to other road users, strand them in dangerous neighborhoods, or hit buses while negotiating road hazards. People will get very impatient indeed if their cars drop them off to go shopping then never return because somebody put a cardboard box in front of the space it’s stopped in.

Some of these problems will be solved with better databases of real world conditions and better AI to analyze unfamiliar situations, but perhaps the most effective way of building these databases and training these AIs will be to utilize the faculties of the passengers who end up experiencing sub-optimal navigation from cars they’re in. “Damn it, Siri, that’s a one way road!” is a very useful piece of data, if the car is equipped to use it. And I’d suspect that a lot of the user feedback is going to sound more like “Whoah, whoah, whoah, WOAH, AAAAAH!”

A truly useable self-driving car is going to have to be part of a complex ecosystem of technologies: hardware, software, automated services, and real-world support. Though companies like Volvo will be doing a good job making sure that occupants are safe while they’re driving, Google has a huge lead in making sure that you’ll be able to tell your car where to go and it will actually get there. Developing an operating system for self-driving cars that integrates the hardware, software, and networked services will be a huge task, and any car company that goes it alone will probably botch it. Lastly, once cars are advanced enough that they’re able to ply the streets unoccupied, or with occupants who can’t drive, such as children, there will still be thousands of circumstances every day where human beings will have to step in and make manual adjustments or judgement calls. Just as you can call for roadside assistance with a dead battery or flat tire today, cars of the future may have to call for a human being to help it understand whether the obstacle in its way is a boulder or a plastic bag, or why it’s lost traction, or what to do when it’s maneuvered into a driveway that now has a gate that won’t open. Over time, Google and Siri will be able to figure out more of these problems on their own, but for a while, we’ll have tech support jobs that involve remotely piloting drone ground vehicles on occasion.