Oye Oyediran has taken his car to the repair shop three times for software upgrades since he bought it last year. This is the future of driving. Cars, planes, household appliances and myriad other machines are increasingly relying on software to work. Manufacturers want the flexibility and innovation that programming code can bring. But software can also make machines accidentally stop working, something computer manufacturers know all too well.
Flaky software could crash your car
2004-01-19 Hardware 28 Comments
I purchased the Motorola a920 mobile phone (on launch day here in Australia), and in my entire life, I haven’t seen a buggier piece of software/hardware. The company 3 (Hutchinson) has lost more sales through my bad mouthing the product (as well as everyone else who bought this buggy piece of shit) then what they spent on advertising. This will come back to bite them, both Motorola and Three. Thats what happens when project managers try to push an unready product to market ASAP.
I will never buy a Motorola phone again, or subscribe to Three for my telecommunications needs.
Developing bug-free software is tough, and some engineering experts believe it might be impossible
This is a load of crap – given enough time (and non moving requirements), engineers can produce a robust, reliable product. Its project managers who push, push, push the unfinished products to market which are responsible for such bad products.
… poorly programmed … programming discipline … strange flaws … There’s always going to be bugs … Testing is part of the problem … the chances of overlooking a bug … pioneered by smart people who didn’t follow the rules … the best of them consider themselves artists or three-point shooters who win the game at the last minute … a breeding ground for bugs … Software is the least disciplined thing in the world … The automotive industry is very reliability-oriented, unlike the computer industry … there’s always going to be a glitch …
Geez, the cause of all these problems is that developers and engineers are NOT given enough time to finish the product. If the project managers didn’t have insane deadlines, always pushing, then none of these problems would exist. Software can be perfect, if allowed to.
I have a motorla T720 and I agree about buggy software. On many occassions i have had to take the battery out to restart a frozen phone.
As for cars, I think some of the code is rushed so that car makers can meet Californias tough smog regulation laws. My Mazda RX-8 can see both better gas mileage and better performance if I choose to buy a 3rd party chip to regulate my car.
If they simply take a little bit more time on quality assurance then their product will do much better marketing for itself.
I think most people don’t evaluate correctly the complexity of todays software project. The number of parts in such projects compared to other industry projects is quickly becoming insane. And yet software engineers are asked to create them and make them interact together well, faster than in any other (successful) industry project.
Some may say there are lots of parts that can be reused in software but that is also true in other industries (they never rebuild a car entirely from scratch..) However, solid mechanics are probably a more stable scientific field than information mechanics.
You’re absolutely correct. Andrew Tannenbaum explains it very well: he points out the contrast between an aircraft carrier, and the software that runs it. Both are enormously complex systems. While the carrier itself is probably an order of magnitude more complex than the software, its much easier to make it highly reliable because its much easier to break it down into components. Even with object-oriented design and reusable code, pieces of programs are just so highly interdependent that its hard to use the “divide and conquer” approach to building software systems.
i don’t like the idea of having software control everything everywhere.
not long ago my car needed to have its control unit fixed because as soon as i reached 160km/h, it slowed down. this did not happen
everytime but that actually even made it worst: it happened several times that i was driving on the highway and all of a sudden the car slowed down. i had to take the car three times to the workshop until they finally
believed me, found the source of evil and replaced it.
and that with technicians, car and software of brand that once was known
for “deutsche wertarbeit” (german high class workmanship).
This is getting real old. People keap thinking a cars telematics are run by the same computers as the actualy cars computers. Its seperate! And authors like this don’t help things. The iDrive system was for asscories, it does not control major systems or safety. Things like brakes will have mechanical linkages for a long time, same as steering, you must have a physical connection by law. Some companies are working on bringing in systems that are by wire for the stearing and breaks for some of the control, but there is still a mechanical fall back there.
Telematics are very seperated from things like your engine computer. I don’t know if it’s just clueless people, or people trying to come up with something to talk about bugless programing, but this whole idea with the cars keeps coming up and is just plain wrong.
You are very wrong about iDrive, it does control the windows and locks. There is one case in comp.risks where the engine fail in a desert region and the iDrive system would not unlock the doors or roll down the windows. Lucky the car was not very far out of town and an outside person who was pasting by smashed a window for them.
And yes, for some reason there were no manual unlocking units – I am sure marketing is behind that also.
As for software, while that is no longer my job, I have written only two bug-free commerial programs but dozens of bug-free personal software. The diffirence is my personal software is written at my onw pace and I can come back to it over years if needed to make it bug-free. The commercial software was always under time pressures and revisions of goals even when the customer had to pay extra for changes.
I wonder why they didn’t mention this:
An example of automotive software gone wrong:
I totally agree with you. It Is possible to write very good software. It takes a lot of time and a lot of well-trained people, which takes a lot of money. Software is a huge case of “you get what you pay for”. When a company wants software quickly and for cheap, the software is gonna suck, end of story.
It is said in the industry that a good line of code costs $200, whatever that means. I have been involved in a project a few years back where I was given enough time to properly write code, and the $200 number wasn’t very far off the mark. In my case the final cost ended up being lower, but I assume that this was because my code relied very little on other modules. It is more expensive to re-write code that has already been written, but it lowers the per-line cost. The surprising part is that for every line of shipping code I ended up with over 30 lines of source file, if you count all the comments, design notes, testing harnesses and unit tests. The code ended up being bug-free, and the few changes that I later did in it were caused by changes in the spec.
As a software engineer, my biggest gripe is the lack of spec, or (even worse) specs that keep changing. After you’ve spent weeks getting your mind designing and implementing an efficient solution for a specific problem, it’s very hard to suddenly change the rules of the game. Lack of spec, combined with lack of time, means that no architecture work is being done, and that’s the biggest issue: you end up with small modules of code that are simple enough to be essentially bug-free, but nobody ever though of how they would interact together, or document it.
Another issue is that writing robust and testable code often results in slower and bigger code. When performance and code size are critical, the ability to thoroughly test the code sometimes becomes a secondary concern.
Finally, I’ve faced several times the problem that QA people are underpaid. This means that the people who have very good training in software prefer to work in R&D than in QA, since those have the choice, and the QA departments end up not being good enough to test code, and only manage to do black-box testing of complete systems which end up being too hard to test.
When I was searching for a link to that old story, I came across an interesting contradiction. In defending Microsoft, Coursey quotes a Microsoft spokesman in this article:
First the problem:
Seems that the computer in Suchart Jaovisidha’s BMW failed, automatically locking all the doors and shutting off the air-conditioning. “We could hardly breathe for over 10 minutes,” the freed minister told reporters. “It was a harrowing experience.”
Then this explanation from Microsoft:
“We don’t have any system that controls starting, stopping, door locks, engine control, or things like that,” said Peter Wengert, marketing manager for Microsoft’s automotive business unit. “We just handle the navigation system.”
In the article here at OSNews:
Oyediran’s car, a BMW 745i, is cutting edge, with an onboard computer called the iDrive that controls hundreds of functions, from the radio dial to the air-conditioning settings.
…Some drivers weren’t initially as enamoured with the iDrive. Reviewers in automobile magazines called the software clunky and hard to learn. Some 745i drivers complained of strange flaws, such as an inability to open the boot from the outside.
Isn’t this acknowleding that the software is responsible for the air conditioning and doors, and not just the ‘navigation system’ as Microsoft and Coursey claimed?
Even this article kind of glossed over it, complaining instead that “it’s nearly impossible to manually tune the radio while driving”.
Tale from the future:
Your car is booting, please stand by….
<40 sek later>
Automatic security update in progress, please wait…
<2 min later, car starts>
Welcome, Mr Steffo!
Well, you aren’t going to take your car if you’re only going to buy some candy. At least *that* would be good for the environment. 😉
I’ve noticed that my new TV takes some 5 seconds from start to show the picture. Is that due to bootup? Are we destined for a future of waiting?
I got a cd920 (this is a couple years old now) and it will switch itself off randomly while talking. Recently, I overheared a conversation of a guy + gal about some mobile-annoyances. So I added my funny experience with my cd920 and instantly he asks: Do you have a Moto..?! lol
The fact that MSFT wrote the OS and/or the navigation application doesn’t mean that they wrote everything that displays on the same screen as their code does. In the embedded world especially it is very common for a single “end-product” to contain code from lots of different vendors.
As a software engineer, my biggest gripe is the lack of spec, or (even worse) specs that keep changing. After you’ve spent weeks getting your mind designing and implementing an efficient solution for a specific problem, it’s very hard to suddenly change the rules of the game
Good use of prototypes might help. No matter how good the communication between the customer and the developers are, it’s hard to understand the spec’s the same way as the customer.
By building prototypes with limited functionality, the users can get a feeling of how you have understood the spec and it’s easier to adjust. By spending “weeks” designing and implementing by yourself, there’s a greater risk that the product will be different that the customer really wanted, no matter how perfect the product conforms to spec.
My family’s new van had to be brought into the dealership when the engine light stuck on. It turned out to be a software problem that they said they thought was fixed in this model. However, there is the alternative of going computer-free (or keeping the same software for years), which means less features, milage, traction, etc.
However, speaking about any device, companies are rushing them out way too fast without great bug testing and fixing. It gets them more money at our expense. That I do not like.
I’ve noticed that my new TV takes some 5 seconds from start to show the picture. Is that due to bootup? Are we destined for a future of waiting?
That’s just the tube being energized. Charging a capacitor that large takes a few seconds when they don’t have an infinite current supply.
They need to stop writing software in dumb-ass languages and employ some computer aided verification tools. Testing can only demonstrate the presence of bugs. Only verification can demonstrate the absence.
I remember a car test in Sweden some years ago (??). It was the small Mercedes A-Klasse. The car rolled over when the driver did a normal turn test, because the anti-roll software was disabled (by disconnecting a wire under the car, something that could easily happen by accident). Baad! The car was outright dangerous to drive without the system enabled!!
Software can be used to cover up fundamental design flaws and that is bad. What will happen when it breaks? It is much better to do a good basic design that functions without to much technology and then maybe add software as bonus features.
A link to the homepage of the Swedish television, that did the test (in Swedish but with pictures of the event):
Another link to some videos. Not certain if they show the roll-over though. Don’t have Realplayer installed on this computer:-):
Actually, in my case the specs change after the customer sees a prototype. We give our customers very early access to our product, far before feature-completeness.
It is for similar reasons that I am happy to have no power windows and no power locks in my car. Sure, I can’t lock all of the doors at once, and I can’t bring all of the windows up or down at once, but I don’t have to worry about the power window motors dying on me or the power locks malfuncting in the bitter cold (which there has been plenty of lately.)
I wish it could be so easy. No sarcasm here. Having been hit by bugs in compilers, CPUs and chipsets, I don’t believe that CAV alone would solve all the software problems to the point where it’d replace testing. If I could simply have a C89 compiler with no known bugs (regardless of whether it was proven to be correct or not) I’d be a happy man.
“You are very wrong about iDrive, it does control the windows and locks. There is one case in comp.risks where the engine fail in a desert region and the iDrive system would not unlock the doors or roll down the windows. Lucky the car was not very far out of town and an outside person who was pasting by smashed a window for them. ”
Windows and locks are not critical systems, they are not connected to the engine computer or critical systems, they are part of the body controller, which telematics can interact with, they are an ascossory (sorry, just can’t spell that word). There is a big differance there.
The solution to software errors is known: Verify logically that the program(s) is following the specification. As JBQ mentions compilers can also be buggy so _all_ programs used in the development process must also be verified.
Problems with verification:
. Programs written in common languages are very difficult/impossible to verify due to weak language specifications.
. Most programmers would not like to work with languages that can be verified.
. The specifications for complex systems are almost always “buggy”.
. It takes more time to properly verify the software (==more money).
. The hardware can fail too. The chance for this to happen can be lessened by using mainframe methods (self verifying CPUs, ECC/parity for all signals++) but is in reality impossible to remove. Hardware support for this is also very expensive.
Would you consider C to be weakly specified? Even if you say that any program that results in undefined behavior or unspecified behavior fails to be verified, assuming that implementeation-defined behaviors and locale-specific behaviors are all documented?
How about not trying to do CAV at the source level (which implies the issues about whether the compiler is correct or not), but instead running CAV on the binary? (after all, all languages are essentially equivalent, right?).
Worse, and that’s the part that scares me, I have the feeling that the specifications themselves could become so incredibly complex that they’d themselves have the be verified through a similar process. But how to verify those?
Even worse, I have the intuition that when systems reach a certain complexity, they become so hard to specify that the only specifications that they can be verified against are so loose that they are essentially useless and that simple code with trivial failure cases would also pass.
Even worse, given a specification, is it possible to prove whether it is possible to write code that follows that specification?
How about specifications that have constraints on execution time?
Its no surprise some software does fail, esp. the way SW is so market driven. But in the PC Software world, we always expect SW to be buggy, crash the computer, etc.
Should faulty software be expected in a microwave, a car, or some other embedded device? It should not and better not. The problem from my viewpoint is the dependence of high level windows code doing the lowlevel firmware work. You can’t hire a person who is a GUI coder and expect him/her to do embedded. Also, interpreters are buggy, which in turn makes it almost a demand for code to be in Assembler. Which to many SW developers is a no-no.
For me, lean code, is the best code. Less supposition of things that can happen, or assume someone else is going to handle it.