I've been developing software for quite a few years. One of the issues that seems to come up again and again in my work is this concept of design and implementation. I recall it being a significant part of my education at the University of Waterloo's Computer Engineering program as well. The message was always the same. Never write code first. First you must design software by writing a design document, flow charts, pseudo-code, timing charts... then it's merely a trivial matter of implementing it. Make note of the attitude here given towards implementing. The real work is in the design, and it's just a trivial matter of implementing it. It sounds so simple doesn't it? Now, how often does this work out in real life?
To read all comments associated with this story, please click here.
Member since:2006-01-10
I've ran into this in the comments here. I almost feel like a repeating record.
Everything you say, I agree with.
The point of the article was NOT to suggest a lack of documentation. I tried to make sure of that in the opening paragraph... apparently, it didn't take.
1. Documentation is needed... absolutely... as is requirements, analysis, high level design... I take these things as a given.
My main point is that software is not implementation. It is all design. As a design, we must take as much care of it as a civil engineer writes his blue prints. Which means taking care of all the stuff you mention with respect to how we treat source code.
The attitude today is what is at fault for everyone. Whereby, people assume the design is all in the design documents. Notice the language used here.
Then the implementation is 'trivial'... can be just done by anyone... the equivalent of assembling a lego design.
The reason why I went through the euclid example is to show that to fully design something, you HAVE TO code it. The code is the design. An English description of a design is good, but English is not the language to handle logic and algorithms. Source code is fairly good. You could use some mathematical or abstract notation as well. But its really just duplicating source code at that poitn.
So handing a design document to a 'code-monkey' will often result in failure as the design document will never specify the full design. The 'code-monkey' will end up doing design (good or bad). We're lucky that many times you end up with a good code monkey who is able to do the design or interpret what is implied by the design.
This is the attitude that must be broken. The source code is the design. There is no implementation. Source code requires just as much vigor as a civil engineer pays to his blue prints. Yes, all the other documents (requirements, high level architecture...) are all wonderful and needed.
However, the final design is software code. That is what the compiler reads and finally outputs.
Hope that clears it up.
Edited 2009-09-11 21:23 UTC
Member since:2007-08-22
<quote>However, the final design is software code. That is what the compiler reads and finally outputs. </quote>
And here is where we - and I say we as others in reply have expressed the same - disagree, and you would by logic have to disagree if you really agree in full (as you state) with what I wrote - because I explicitly wrote the opposite, as have others.
The source code of software IS the implementation of a design - that is its nature, since you can implement the design of a program in numerous languages and still have the same program operating the same way - whether you implement it in Java, C, C++, Objective C, Assembly, or YAL (Yet Another Language - whatever you please).
While your teachers proceeded under the false assumption that the implementation is trivial - most don't. Indeed, good teachers would not as they themselves realize that as with Civil Engineering the implementation is no more trivial than the design itself. Indeed, with Civil Engineering you have many people working on the implementation and many checks to verify that they are all following the design to the letter, and even more so - following the standard procedures that are not part of the design but assumed by the design - assumed by the Civil Engineers that wrote the design.
Conversely, in software there are extremely few standard procedures that anyone that only performs design can assume - thus the design must be clear all the way down to the source level, and sometimes further than that - explicitly stating what the machine should do at the machine level. (Indeed, some systems require such accuracy.)
However, as stated by myself and others, source code only tells you what the software actually DOES, not what it was INTENDED to do.
Perhaps you need to think a little differently -
Source Code is the implementation of the final software specifications - that is how Traditional Software Engineering sees the source code, however wrong it may be (and I do agree that view is wrong).
The primary problem at all levels in software engineering is that the field has yet to formulate standards, and software developers are to arrogant to follow what has been laid out - they want to do their own thing regardless of what anyone tells them. So there is a great lack of discipline in the field.
Conversely, when a bridge builder is helping to build a bridge there are only certain ways in which the builder is allowed to put a rivet in, certain methods of how a weld is allowed to be done, etc. And for instance, many of these are verified by others - a welder does not simply weld and go away; rather the weld is inspected and verified that it is to specification required by law and by the design (since the design usually exceeds the law). The inspection may primarily be done my a master welder, but the welder is ultimately responsible for any failure in his work - if the weld fails and someone dies, the welder and (even more so) the weld inspector may be charged with criminal negligence, and the company that built the bridge would be in court for wrongful death. There is liability attached.
Now consider software implementors, and designers. Are any of them liable for failure to do their every day job? Aside from being fired, there are few that are - namely those that are involved in fields of life and death, but even then it is likely rare the software coder would be liable beyond being fired.
Does anyone inspect the code to verify it against a design? Does anyone verify that it properly handles failures? Do most programmers implement not just units tests but also functional tests? Are most programs even testable to be verifiable?
Sadly, the answers are mostly a resounding 'NO'; and most of this comes back to a lack of even basic discipline in the software field, and the arrogance of software coders such that they refuse to do anything in any way other than their own - especially when it comes to coding styles.
So while you call out for documentation, you primarily fling it back in your analysis as something that is not required since the code should be documentation enough - and that is not the case. If that is not what you meant, then you failed at such communication - both in your original set of articles and in your reply to my original post.
Traditional Software Engineering has many faults; faults which need to be corrected. But the culture in which software is produced also has many faults that need to be corrected. And software will continue to flounder about as it has until these are both resolved.
But there is one other thing to remember - they must be resolved such that they are practical for software coders and the companies that employ them, and this is a great part of the failure of Traditional Software Engineering, which is also partly (but not wholly) attributable to the arrogance of the software coders as well.
Sadly, I think while you hit on the nail of one side of the problem, you also fail to realize the whole of it, or even the entirety of what you wrote.
And with all respect, I sincerely ask that you sit back and think a lot more before writing on the subject again - think about the implications. I would also be willing to work with you more on it, and if you so desire then send me a message privately and we shall go from there.
Member since:2006-02-26
I appreciate the interesting discussion here, but I respectfully disagree with this. From my point of view, the problem is the attempt to pigeon-hole the practices of traditional engineering into software development. We don't need more discipline. We need less.
There is a major difference between someone who is a programmer and someone who has a job as a programmer. It takes years of dedication and a tremendous investment of personal time to become a good programmer. Those types of people simply aren't going to accept (or if they're forced to, they certainly won't respect) jobs where their primary task is to translate 10,000 pages of poorly worded specification into code. So the people you're left with to fill that role are those who barely made it through a CS program and have no other option but to accept the job. It's not surprising that the resulting implementation is typically poor.
Give the programmers more freedom; give them more responsibility. Throw out the ideas of requirement gathering and design as separate and disconnected phases and give the programmers-- the ones actually implementing the code-- a direct, open dialogue with the clients. Allow programmers to do what they love to do: solve problems. You'll attract more competent programmers to your company and produce better code as a result.
As long as the practices of software engineering continue to view programmers as laborers, akin to someone who drives a rivet or welds an I-beam, the process will continue to fail. If we desperately need to map traditional engineering onto software development, I'd suggest that the best analogy is that the programmers are the architects and the compiler represents the labor force.
This is not to say that the code is the specification. I've been involved in enough large scale projects to recognize that up front design is supremely important. I only suggest that insulting programmers from that process and then expecting them to be code monkey translators is a recipe for failure.
Member since:2007-08-22
To some degree, a very well agree with you, and you simply misunderstand by what I mean by discipline.
To start with, as with any other job, the only people that really going to be good at the job are those that truly love it and are dedicated to it - this holds true whether you are a shop keeper, a welder, a CEO, a programmer, or even (dare I say) a politician. On that point we agree.
However, you don't see the discipline in which I speak - you mis-associate so what you are thinking is different from I am thinking. So please bear with me while I layout my thoughts, and try to make them as clear as possible at the moment - at least those which I have thought enough to be able to well formulate.
So let us first layout what I mean by discipline:
To start with, a programmer must exercise great discipline in how a program is laid out. Logic should be clear, predictable, well documented (e.g. comments in the code that are clear). Coding style should be uniform across the team and the company as a whole; that is not to say that the programmers would not work to build a single coding style, but once laid the MUST follow it, leaving their arrogance aside. (A hard task, but it is a must if software programmers are to be respected.) Errors must be handled - ALL errors, not just those that are expected.
While there is more I can say, I have more thought to do before saying it aside from what else I will answer while responding to your comments below.
I very much agree that programmers need more freedom; but not likely on what that freedom is. For instance, coding style is not a freedom for the programmer - to a company, programmers come and go; but the programs remain, and another programmer must be able to pick up the work and continue it on. Programmers need to realize that they are not the only person to work on a project and that they must build the project such that the next fellow can pick it up - the only assumptions should be that they will be familiar with the field in which the program is targeted; that is - if the program is specific to a field of biology, the comments should reflect that and speak to that field so that a programmer (who may not be quite as experienced as the one writing the code) may be able to pick up the software and continue. Likewise, the programming style should be uniform so that any programmer working at the company can quickly pick up the program's source and be able to understand it - having similar logic, error handling, etc; allowances of course as implementation languages dictate.
However, I very much agree that the programmers - the technical people - need to be involved in the requirements gathering and design - e.g. the concept of operations, analysis, architecture, requirements, architecture, etc. The earlier they are involved the better - and not just one programming individual, the whole team in so far as who is working on the project. (E.g. if developer A is not assigned to the project, then they should not be in the meetings; but once it is realized that developer A will be on the project they should be immediately brought in, brought up to speed, and made part of the effort from then on.)
Let them talk to the customer - or at minimum, let them listen in directly on the conversation - e.g. they're in the same room during the discussion - even if the company wants all questions to go through a specific individual who ultimately talks to the customer; and give them some way to pass questions to that individual on the fly.
I realize that most on this list will not like many aspects of the discipline I propose - and for that, I simply would have to reply - grow up, humble yourself, and learn to work as a team for the company. Creativity can still be had within those limits, and far more will be doable and achievable. And if I were your boss and you wouldn't follow the policies set forth, you'd be out on the street just like anyone else who wouldn't follow the company policies - following such policies is part of your employment contract. (And yes, it is legal to let someone go on those grounds.) The sooner you realize that businesses are less and less going to let programmers get away with the antics they have in the past, the better - because it is changing.
Instead of thinking that your job security lay in that only you can understand the code (it doesn't - it never will), think instead of your job security as the ability for you to bring profit to the company - the more you can bring, the better. (And yes, that can be with FOSS too!) But it first starts with working to achieve standard processes, policies, etc. for writing software and then following them and ensuring the whole team follows them.
And if you don't think this works - just look at the Linux Kernel. No patch is accepted unless it meets the standards set forth by Linus and his band of lieutenants - standards that even layout an explicit coding style. It works; and it can be achieved even among the most creative.
There is a time for being creative, artistic and there is a time for it not. Learning to differentiate between them is a good software engineers responsibility; and doing so shows a great maturity that will garnish respect. Coding style is one of those times to set aside your creativity and artistry and just get with it; your work life will be better for it too.
And, btw, I do my best to do this myself. I have one preferred coding style; but I do my best to follow what has been set before me. I may use my own style on projects (it does fit the guidelines of my company - the company coding style is too inclusive to start with, but that's all I shall say on that), but I also try to fit code for others projects in the same coding style that project uses. In other words - I try to adapt to what the company requires of me, no matter my genius or lack thereof.
Member since:
2007-08-22
There are several problems with the proposed approach, and I won't re-iterate the details of what has already been said - just summarize and add:
1. Code may be incomplete too; it only tells what actually DOES happen, not what it was INTENDED to do. You can no more read the mind of the coder than you can the specification (design, architecture, concept of operations, requirements, etc.) writer.
2. Specifications are NEVER complete. You can't necessarily capture all use cases in the specification. One implementation language may have use-cases another doesn't, and the specifications writer may not understand the language specific implementation details.
You also start with the assumption that THEORY makes the program. Suffice it to say, it doesn't; and the Academic world is killing the prospects of the students for it. Academia focuses way too much on theory, and no where near enough on the practical implementation. Businesses care more that a programmer can (i) understand the concepts presented, and (ii) implement that concept than they do that the programmer can theorize about the concept and vaguely tell someone else how to implement it. They are in business to MAKE money, not pay multiple people to do what one person could have.
However, that still doesn't get down to the root of the problem you are trying to solve. The biggest problem in programming, aside from Academia - though Academia is also partly to blame for it, is not so much theory and implementation, nor how programmers are taught to program, but rather the lack of discipline in the field and the arrogance of programmers themselves.
To start with, a lot of programmers write software as if they are ever the only person who will ever, ever touch the source code for that software. But that is absolutely wrong - most programs are initially setup by either one person or a small team, and then quickly replaced by others who then have to maintain it. Whether they initial writer leaves the company and goes somewhere else, or simply moves onto another project - it doesn't matter; someone ELSE has to pick up the ball and carry it. And sadly, that someone else does the same thing.
Additionally, as programmers we more often than not fail to get even half of the user's requirements before we make assumptions on how the program should operate. And then not only are our managers surprised when the user doesn't like (or even use) the software, the programmers are too!
Needless to say, I think most all of the problems in software can be brought down to a matter of the lack of discipline in software writing, and the lack of humility of the programmers to follow even basic coding styles as a team.
Discipline must be followed in every step of the way - from gathering thorough requirements (as much as the user can give, not as much as it takes for the coder to decide what to do), to handling error conditions at every possible point (it's far more robust, and C++/Java Exceptions lend no help to this matter; that's not to say they are not a useful way of communicating errors, but those errors must still be handled at every possible level in the program, not simply caught and discarded, or left to a higher level to find), to design of interface (both HCI and CCI), to documentation IN the code, and specifications writing (at all levels - concept of operations, requirements, design, etc.) There's more, and hopefully I'll someday be able to publish my full thoughts on the subject to OS News and Slashdot, but it won't be today. (I'm partially still gathering my thoughts together to write, and also need to write what I already have more thoroughly; but it will happen in time.)
It's obvious from the article that the writer missed the point of design; but they probably also missed (likely entirely) the stages that PRECEDE design - concept of operations, requirements gathering, and architecture. All these stages are important, and all, as with those that follow them, are iterative. But none of them solve the end problem - stable, secure, robust software - because all fail to capture everything. These stages (design and implementation included) are what I call Traditional Software Engineering, and the only people that can afford to do so are those who are putting lives in danger (e.g. DoD, NASA, etc.) and cannot afford even the slightest error. (That said, DoD Contractors don't do it quite right either - they only do as much as the Gov't Program Manager makes them, who only does as much as the Gov't budget allows.)
There has to be a better way that makes it practical, and being more disciplined about software writing at every step of the way is at least part of it.
(Obviously, I'd love to hear other's ideas on what it means to be a disciplined software developer.)