I've been developing software for quite a few years. One of the issues that seems to come up again and again in my work is this concept of design and implementation. I recall it being a significant part of my education at the University of Waterloo's Computer Engineering program as well. The message was always the same. Never write code first. First you must design software by writing a design document, flow charts, pseudo-code, timing charts... then it's merely a trivial matter of implementing it. Make note of the attitude here given towards implementing. The real work is in the design, and it's just a trivial matter of implementing it. It sounds so simple doesn't it? Now, how often does this work out in real life?
To view parent comment, click here.
To read all comments associated with this story, please click here.
Member since:2006-02-26
Ah, things are clear now. We do agree on the problem and on the desired outcome, but we disagree on the solution. To put it simply, you argue for a tighter leash on programmers while I argue for a looser one.
I believe that applying engineering disciplines to software development was a mistake from the start and to further entrench those methodologies would only serve to exacerbate the problem.
I'll present some of my reasons. I've given this quite a bit of thought over the last few years, but like you, I haven't spent a lot of time formulating this into a coherent set of arguments, so consider this my scratch pad.
First, building codes rarely change and when they do, it is usually by very small increments. This is typically the result of environmental factors, the development of new tools or materials, or the discovery of a defect in a current process. Not to mention the fact that any sort of structural engineering is very conservative by default due to the loss of life or property that is possible in a failure condition. So changes in traditional engineering happen at an absolutely glacial pace compared the rate of churn in new ideas, technologies and methods in the software industry. This presents a major stumbling block for generating any sort of standardized "language" for software development that would do anything more than hamper the agility of company that subscribes to these standards while trying to compete with another that does not.
We agree that UML has already proven to be an abysmal failure. You might also agree with me that UML failed because it is incapable of expressing sufficient detail. I would argue that too little detail leaves your design open to error and misinterpretation but that too much solidifies it in the face of changing requirements. I don't believe there is a "sweet spot" to be found due to the chaos that is inherent in the field.
To put it succinctly, building a bridge today isn't going to be all that much different from building one ten, or even twenty years ago. Software development, on the other hand, has changed drastically in that time.
Second, the nature of programming is fundamentally different than that of construction. A welder working on a bridge is not required to understand things like variances, setbacks, wind speed tolerances or load distribution. That is the job of architects and engineers. The line between design, engineering and labor is extremely clear cut. The same is completely untrue when it comes to software development. A competent programmer will understand the project that he or she is working on, from the high level design all the way down to the machine code that will be executing, often to a greater extent than those that originally designed it.
Yet software engineering attempts to apply the same design/labor dichotomy to the software development process. I'd argue that this forced dichotomy engenders recalcitrance in programmers due to the fact that it squanders their skill, devalues their creative input, and effectively reduces them to specification/code translators. As we've both agreed, a programmer becomes skilled through their love of the trade, through their joy of using a computer as a tool to solve problems. When you take the ability to solve a problem away and reduce them to mere labor coders, their job satisfaction and motivation to produce quality code will quickly drop to nil and they'll do the least amount necessary to take home a paycheck. Or they'll quit and find another place to work where they feel they can actually exercise their knowledge.
Third. Well, I don't have a third yet. The ideas are still bouncing around and I don't want to ramble on for too much longer.
Suffice it to say that I think burying programmers under stricter requirements, tighter management, government certifications, and formal languages will have the exact opposite effect of what you intend. Instead, loosen your management, set the programmers free, put them in charge, and they'll produce for you. If you disagree, take a look at Google. It has a very relaxed management structure, attracts some of the brightest minds in the field, and produces some of the most successful software on the planet, often ahead of any announced schedule. This method is not an anomaly-- it is the way forward and any companies that don't follow suit will continue to face missed deadlines and project failures, and will eventually be run out of the market by their more agile and motivated competitors.
Edit: Broke up some rather large paragraphs. We may be able to co-author a book by time this thread is finished.
Edited 2009-09-12 18:44 UTC
Member since:2007-08-22
You misunderstand what I mean by discipline.
Traditional Software Engineering - and even present thought on Software Engineering - is dead wrong. It doesn't work, and it's not practical - too laborious and too costly.
Rather, I am arguing that certain aspects of Traditional Software Engineering are good, and helpful (e.g. Concept Of Operations, Requirements, Architecture, Design, etc.); but not necessarily as they are now. They are good and helpful namely because they force communication to occur so that both sides understand what is being delivered in plain language. They also provide something both sides can test against and say without doubt that what was agreed to has been delivered.
However, while the "art" surrounding those documents is pretty much already settled - since it applies to far more than simply software - there is a whole other side to engineering that software presently lacks - and that is simple discipline in the field, of which those documents are merely a part.
Software today is (i) highly unreliable, (ii) highly buggy, (iii) a support nightmare, (iv) very costly to produce, and (v) late on delivery if it is even delivered. Most software projects - even small ones - are vastly over budget; and the majority fail ot even make it to delivery. But that need not be the case - software CAN be reliable, with few bugs, on budget, and on time; but we don't get there without discipline.
I'm talking about discipline similar to how one is disciplined about driving a car. There are certain norms - e.g. drive on the right side of the road, etc. You need those norms to provide a good experience, and do what is necessary. With respect to software, the programming language IS NOT one of those norms. Rather, the norms apply to the coding style, how the logic is layed out, etc. In other words, how the programmer does the job of programming. They need not be overly burdensome, but they do need to make code more uniform at least at the company/team level. (I say company/team because when working for an employer, it should be defined by the team of programmers working for the employer - all of them; however, when working on a multi-company project or on a project not related to a company, then it must be defined instead by the team of programmers working on it.) By providing uniformity, more people are more easily able to spot the problems.
Fair enough; I think I may be a little further along than you, but not necessarily by much. I'm all ears, and I'll certainly try to incorporate what I've learned into what I am working on.
Okay, you actually have two multiple things going on here. First, I very much do agree that software is a very different beast than any other engineering field; first and foremost because it is in a very different environment than any other engineering field. In all other engineering fields you have very exact physical limits - e.g. the law of gravity, the structure of a cell, etc - where as with software, you have to create your universe nearly from scratch for every project, well not quite any more since we mostly write for one or more operating systems that do provide some continuity in the environment. So to compare - civil engineering is predictable because the laws of physics never change; but software is far less predictable because the environment is constantly changing (e.g. what other software is installed, the hardware it is run on).
Building Codes equate more equally to programming languages. They don't come very often, and they seldom change.
You get the structural engineering part write, but still fail at the software engineering side.
There is nothing prohibative about writing a standard set of procedures on how to program. I would not say that any programming language itself should be standardized upon - indeed, no single language works for every instance of application implementation. However, procedures for writing a program can be.
For instance, consider how one implements an algorithm. Do you check the error condition first? Or do you check the success condition first? How do you handle the errors? Do you check inputs to see if they are valid? Do you check pointers? There are a lot of little details like those that can be very easily standardized upon, details that after being decided upon aid the programmer in keeping from creating bugs in the software, and they do not prohibit the advancement of software development in any manner - rather, they promote it.
Again, going back to the questions I asked in the last paragraph, think of how many different answers you can come up with just from that small set of questions, and there are many more that need to be addressed. Standardizing on things such as these will enable coders to do more - code will be more quickly written, and more easily verifiable - not just be tools, but by people as well.
Furthermore, you increase what I call your "Bus Factor". That is - if you are the only one working on a given piece of code on the project, and you were to get hit by a bus today - (i) would someone else be able to pick up your code and continue, and (ii) how quickly would that be able to happen? By standardizing, you enable someone else to be able to pickup what you are working on, and continue it, and you decrease the time it takes them to figure out what you have been working on. (BTW, I've done just that 3 times already - first time for someone who was fired, second time for some that literally died, and the last time the guy quit.)
Programming such that you are the only programmer to be able read and understand the code is NOT professional. It can be guaranteed that you will not be the only programmer on the project; so humble yourself and write your program with that in mind. More likely than not, the next programmer won't be as smart as you so it needs to be clear enough that someone with lesser experience can pick it up - they won't be able to read your mind, especially if you got hit by a bus (e.g. died).
Very much agreed.
UML has a strength - it can represent something in a graphical way. However, it's greatest weakness is change. Software changes rapidly during development, and UML will quickly become outdated. Even if the initial UML model was able to express every detail necessary, by the time the coders were able to get just the initial coding done it would be outdated - things would have changed - there would be more functions or less, more classes etc.
[q]I would argue that too little detail leaves your design open to error and misinterpretation but that too much solidifies it in the face of changi
Member since:2006-02-26
Your post apparently got cut off, but there are a few clarifications to be made so I figured I'd toss them in now.
I specifically put "language" in quotes to imply that I was speaking of a formal language for specifying the procedures of a software development project. Not a programming language. Sorry for the confusion.
Like I stated, we do definitely agree on the desired outcome. That formal procedures for code style, layout, organization, error handling, etc. are established and followed throughout a project cycle by all team members unless a specific situation calls for deviation, in which case it is clearly documented.
Our disagreement lies in the methods of achieving a team of disciplined programmers who follow these practices.
I didn't notice a response to my statements about Google. They may have been part of the post that was cut off but I feel they are pertinent to the discussion so I'll reframe them as a series of questions here.
Why does a company like Google seem to have a surplus of great, disciplined coders who can pull off successful projects ahead of schedule?
On the other hand, why do in-house and contract development projects consistently fail due to lack of discipline?
Why continue to push software engineering or variations thereof when, as you've mentioned, it has so far led to nothing but unreliable, buggy, costly and late software and other models have already proven successful?
I have my own answers to these questions, but I'm honestly more interested in yours, as it seems you've spent at least as much time pondering this subject as I have and I always enjoy differing yet well-informed views.
Member since:
2007-08-22
Well, I had written alot more, and then lost it.
But basically:
I think we agree very much.
As with other engineering disciplines, when the Software truly becomes a discipline, the geniuses will be able to excel while the standards will keep the slackers either from the field entirely or at the very least, from doing too much damage.
I find Traditional Software Engineering, and "modern" software engineering to be wholly insufficient and utter failures at the task - but again, I think this is partly to blame because of the lack of true discipline in the field. Software Engineering only works when there is a basic level of common understanding on how to do things - a common language that all understand, a common methodology for how things work, and a common level of competence such that failures and successes can be learned from.
In other words, the software field is presently very chaotic with everyone wanting to do their own thing. Some work out, but sheer luck; while most fail. In developing standard practices, and a true discipline to the field - one that is taught as rigorously as any other engineering field is - then we should be able to turn the tide, and instead of the majority of software projects failing (as is today), we can have the majority succeeding.
When the software field is disciplined as such, THEN we can certify software engineers much like we do any other engineering field. Where as today, we have one U.S. State (Texas) that does so, and part of is simply testing ones knowledge of UML - something which most use, and is truly useless to the software field. A certification of engineering is only useful when it can test standard practices, but the standard practices must first exist, and today they do not. Hopefully, by the end of my career they will - I at least certainly aim to my part to ensuring they do by then.
As Software Engineering is presently defined, I agree. However, I think it is defined incorrectly, and a proper definition will bring out the best, especially in the good ones who would be able to excel even more.
And if you think I am wrong, consider how well other engineering fields still maintain their superstars - how people shine in those fields, and how respected they are. Now contrast that today, with how the "best" of software are seldom respected, and mostly found to be arrogant - there are exceptions, but they are few.
Managers typically hate software developers, because they find them unmanageable; and that is primarily because of the lack of discipline in the field which directly leads to a lack of control of the projects. Time estimations are presently worthless, but with proper discipline can actually be done with little error - and that is what management wants, and what helps make a profits for the company.
Such discipline would also benefit both open source and commercial software projects. Not just one or the other.