However that doesn't mean I completely disagree with Lamport's arguments. Specifications are necessary in large-scale software because otherwise you can't know if you have actually implemented everything the program is supposed to do. When you release it the end-users get angry and insist that they told you in that email / meeting / watercooler conversation last year how important it was that the program frobnicate the warbles.
Lamport makes much of formal specifications. I agree that in some cases they do add value; trying to prove the correctness of a distributed protocol in any other way is dumb. But in my experience once you get up into application-level requirements a formal specification is basically source code you don't know how to compile; there is no formal correctness criteria against which you can verify it, and the stakeholders can't understand it in order to verify it.
The "source code is the blueprint for software" analogy doesn't work because the source only tells you what the application does, not what it's supposed to do. There is no way to tell, from the source code alone, whether or not what's been built is fit for purpose. You have to have some other document. The "specification is the blueprint" analogy is saying that the specification is a high level document that sets out what the application is meant to do.
As an example, imagine you're paid $1m to assess my super awesome new database. I give you the source code and tell you "the source is the blueprint." After reading through the code you discover there's no DELETE function. Have I built a clever immutable data store, or am I just an idiot who forgot a critical function? You can't tell that from the source.
"TL;DR: Source code is the blueprint for software."
...and the 2x4s are the blueprint of a house and
the steel beams are the blueprint of the skyscraper....
I'll take the TL;DR at face value since it was written by the author. The statement completely obliterates the entire concept of a blueprint or plan. By that reasoning, it is true everything is its own blueprint, and the concept has no meaning (of course this is a standard tautology).
And, simply nodding to CYA documents (for when the customers/end users come back saying that your product doesn't do X), or to docs for when formal proofs are needed, doesn't cover the concept.
The point of planning (or blueprinting) is to think about what you are going to build before you get out the tools, abstracted away from the specific tools that you'll use, and to then record and communicate those thoughts to all the stakeholders. It doesn't have to be a complete formal, engraved-in-stone blueprint, but a modest amount of actual thinking and planning goes a loooong way.
I've often found that with a little thought, we'll use a different tool than originally expected. If you expect to just dive in and sort it out in your source code, then you've already a priori restricted your choice of tool, and probably also your library, algorithm, etc. (of course if you only have or know one tool then, well, 'everything looks like a nail and who needs a blueprint?').
Without planning, you'll get what grows out of the first thing that comes to mind, not the best thing you could build. The mere fact that it manages to run or stand is not a defense of the bad process.
Planning is where the real thinking takes place. Sure, if all you want to do is exercise your programming skills and avoid thinking, then don't plan. But don't pretend that no one else should think or plan, or complain when someone else presents a plan to which you are expected to build (and don't be surprised 10 years on when you don't advance as far as you expected).
But the article directly says source code can't be the spec because it contains low level details. E.g. you implement something with map/apply instead of loops because that's all your programming language has, or your filter out some characters from a data stream because they often make a third party parser raise Exceptions you don't understand. These are programming details that the source code contains but the spec shouldn't. Therefore I think your argument is debunked by the main article already.
In any case it's a good discussion to have in this thread, which is why I gave you the up.
I was going to go the other way here and point out that blueprints are actually code for buildings.. the builder is the compiler I guess?
But blueprints are done at various scales and they differ in complexity and format ie. sketches, models, schedules. They also often come with a book of specifications, and these are typically only referred to when ordering parts or if something goes wrong - and they are always open to interpretation..
Yes this is exactly how my organization works, because my team leader is too lazy to spec anything out.
Its an absolute nightmare when something doesn't work as expected, because rather than checking a specification that will take a couple of seconds to read, you basically have to debug software, which takes a lot longer.
TL;DR: The real problem to solve is the time to iterate.
I think it's a straw man argument. An unfair comparison. Buildings & bridges etc have been designed and improved for literally thousands of years. Software is a half a century old.
Aza Raskin's blog piece about human powered flight which bubbled up on HN has always stuck with me[0]. "The problem is we don't understand the problem".
I could make the argument that source is the blueprint for a program. You use blueprints to know how to build a building, and you use source code to know how to build an executable, or rather, have a tool that automatically makes the program from the blueprint (source code).
What I've seen before, and seems to be the case in comments here, is that beliefs about programming methodology are heavily influenced by what kind of programming a person does. Those who work on small projects (i.e. few parallel streams of development) love their agile/scrum/whatever. Those who work on larger projects are less sanguine. Likewise, front-enders also love the hipper methodologies, while back-enders tend toward the old school, independently of whether their projects are large or small.
The confounding case is the Linux kernel, which is both large and systems-focused but seems to eschew both kinds of methodology in favor of undirected chatter on mailing lists. Then again, maybe that's just part of a two-level process. More than a few patches to the kernel have real specs behind them, though developers don't call them that. They're just "kind of long, kind of detailed" emails. This tendency to embrace a thing while rejecting the term for it is one of the weirdest tendencies many developers seem to have.
I know the author's entire argument doesn't stand or fall with the simile, but it still needs to be said: blueprints are more necessary for physical construction than they are for software because things made of wood and concrete are hard to change part-way through.
It is very important to differentiate between House and Skyscraper. You can't build a skyscraper (currently) without blueprints. But you can, and it has been generally done, build houses without blueprints. It was even common to build cathedrals without formal blueprints, by the way.
I have built a house. You don't necessarily need professional blueprints but it's a good idea to at least have a scale drawing of your floor plan. Otherwise you won't really know how much of various building materials you will need. How many windows? What size? Where will they be positioned? Will they actually fit there?
Also from a practical standpoint you probably won't be able to get a building permit without blueprints or drawings of some sort (though this varies).
Yeah but they are done before you actually start to lay any bricks.
In software you most of the time have the 2 things running parallel because of time constraints and other business needs, which almost always leads to failure.
Once again I must refer to Naur's "Programming as theory building". Effectively programming is implementing a specific model. For the outcome at first release it does not matter how the model is documented - just that it is understood by the team. As time goes by, the more obscure the model is , the harder it is to maintain the understanding within the team.
This probably one of the reasons some agile practitioners scoff at documentation - if one is using the same framework to implement the similar website time and again effectively the model remains the same and understanding of the model is easy to retain and retrain.
> But few programmers write even a rough sketch of what their programs will do before they start coding.
deserves a reference to a study. (or, and this is the first time I use this word: a #strawman flag.)
- or am I the only one who sketches up data models and data flows before going to work?
Also: Prototyping comes early in many other designs as well, often way before detailed blueprints are signed off I think (not a mechanical engineer, just outside observations).
In software it just so happens that prototypes are often made by the same people and from the same material as the final product, only reinforced to withstand production load and abuse.
> am I the only one who sketches up data models and data flows before going to work?
Anecdata: Yes, or rather you're one of a vast minority. I work in an office with no notebooks in the supply closet, and one 5x5 whiteboard which is usually full of UX layouts.
Though the idea on specs vs proof systems is good enough , one of the reasons type systems are being studied is to actually make programming languages themselves a good proof system , so much so that DSLs themselves become the spec.
Proof systems have their uses and then there are places where they are just too much.The best is in the mid!
>> "am the creator of the TLA+ specification language."
I think TLA+ is/was being used at Amazon [1]
Safety with Preservation and Progress guarantees of critical systems demand design audits at a larger scale however if the scale of the program is actually quite small , things like type-systems themselves are sufficient proof of safety.
Ofcourse there are cases where even typing is too much!
One of the middlepaths is actually using inspection tools on code/programs as another aspect altogether.
I think the main thing missing in the discussion is organization or team size. Given that the author is a CompSci researcher for the past 40 years he may not be in touch with the reality of how organizations are structured and the impact of that on software development.
If it is a start-up with 5 people, the software probably is the specification. If it is a large organization of devs (25, 50 or 100 people), then you will see more documentation, some of that specifications.
There is likely a threshold in the number of people where the number of communication paths between them is too large to rely on informal ways of communication. Maybe its 15-25 developers. The larger an organization is the more bureaucracy gets introduced and you now have people whose role is to write specs and requirements docs, etc.
Somewhat off-topic anecdote on this: my father in law designed a very rough draft of "blueprints" for his parents new house in rural China (Xinjiang) using MS Paint, local labourers built it in a few months and it's still standing 10 years later!
From a European perspective, I like how you say that the house is still standing 10 years later (and someone in another comment above mentioned 25 years) like it's something outstanding.
I would expect a house to last 100 years. A family member once bought a house that is over 400 years old (and still standing).
Then again, construction and houses are way more expensive in Europe/Germany. They're planning to tear down and rebuild my old high school from scratch, current estimate: 85 million euros total.
"Programmers spend a lot of time thinking about how to code, and many coding methods have been proposed: test-driven development, agile programming, and so on. But if the only sorting algorithm a programmer knows is bubble sort, no such method will produce code that sorts in O(n log n) time. Nor will it turn an overly complex conception of how a program should work into simple, easy to maintain code."
I wouldn't know a bubble sort off the top of my head (I haven't had to implement anything like that since university). I would know to read up on different algorithms and choose an appropriate library based on what I was sorting. (It will probably be better tested than what I would come up with first time).
While I do think proofs and planning are both useful, I think that the analogy doesn't work well.
The process of producing code is the process of specifying what the computer should do -- the deliverable is the specification. On the other hand, few civil engineers will start designing plumbing without at least having a sketch of the floor plan and an idea of the building uses.
The process of producing code is the process of specifying what the computer should do
I'm sure coding is "specifying what the computer should do", but that's not what people are talking about when they talk about specifications. A specification defines what the application should do. Producing code is implementing the specification. Without having any specification first you wouldn't have a clue what the code needs to do, so you wouldn't know what to write.
The problem with the analogy is that most people don't understand what a blueprint actually is. A blueprint is an architectural planning document. It defines the building in very broad terms - where the walls go, where there's a staircase, and so on. It doesn't go in to the detail of how the building is constructed. That comes from all the other documentation. As an analogy with software it's a pretty good one - a building's blueprint is equivalent to the high-level planning documentation that any big project needs.
Off topic: my house was built without blueprints. Talked to the contractor, sketched what we wanted. He built it. Standing 25 years later, great job.
An experienced carpenter/contractor can build a house, without excruciating detailed notes. In fact they can build it faster, adapting to conditions as they change and solving problems efficiently.
[+] [-] paulajohnson|11 years ago|reply
http://paulspontifications.blogspot.co.uk/2013/01/software-h...
TL;DR: Source code is the blueprint for software.
However that doesn't mean I completely disagree with Lamport's arguments. Specifications are necessary in large-scale software because otherwise you can't know if you have actually implemented everything the program is supposed to do. When you release it the end-users get angry and insist that they told you in that email / meeting / watercooler conversation last year how important it was that the program frobnicate the warbles.
Lamport makes much of formal specifications. I agree that in some cases they do add value; trying to prove the correctness of a distributed protocol in any other way is dumb. But in my experience once you get up into application-level requirements a formal specification is basically source code you don't know how to compile; there is no formal correctness criteria against which you can verify it, and the stakeholders can't understand it in order to verify it.
[+] [-] onion2k|11 years ago|reply
As an example, imagine you're paid $1m to assess my super awesome new database. I give you the source code and tell you "the source is the blueprint." After reading through the code you discover there's no DELETE function. Have I built a clever immutable data store, or am I just an idiot who forgot a critical function? You can't tell that from the source.
[+] [-] maxPwrWgt|11 years ago|reply
...and the 2x4s are the blueprint of a house and the steel beams are the blueprint of the skyscraper....
I'll take the TL;DR at face value since it was written by the author. The statement completely obliterates the entire concept of a blueprint or plan. By that reasoning, it is true everything is its own blueprint, and the concept has no meaning (of course this is a standard tautology).
And, simply nodding to CYA documents (for when the customers/end users come back saying that your product doesn't do X), or to docs for when formal proofs are needed, doesn't cover the concept.
The point of planning (or blueprinting) is to think about what you are going to build before you get out the tools, abstracted away from the specific tools that you'll use, and to then record and communicate those thoughts to all the stakeholders. It doesn't have to be a complete formal, engraved-in-stone blueprint, but a modest amount of actual thinking and planning goes a loooong way.
I've often found that with a little thought, we'll use a different tool than originally expected. If you expect to just dive in and sort it out in your source code, then you've already a priori restricted your choice of tool, and probably also your library, algorithm, etc. (of course if you only have or know one tool then, well, 'everything looks like a nail and who needs a blueprint?').
Without planning, you'll get what grows out of the first thing that comes to mind, not the best thing you could build. The mere fact that it manages to run or stand is not a defense of the bad process.
Planning is where the real thinking takes place. Sure, if all you want to do is exercise your programming skills and avoid thinking, then don't plan. But don't pretend that no one else should think or plan, or complain when someone else presents a plan to which you are expected to build (and don't be surprised 10 years on when you don't advance as far as you expected).
[+] [-] erikb|11 years ago|reply
In any case it's a good discussion to have in this thread, which is why I gave you the up.
[+] [-] cam_l|11 years ago|reply
But blueprints are done at various scales and they differ in complexity and format ie. sketches, models, schedules. They also often come with a book of specifications, and these are typically only referred to when ordering parts or if something goes wrong - and they are always open to interpretation..
[+] [-] collyw|11 years ago|reply
Its an absolute nightmare when something doesn't work as expected, because rather than checking a specification that will take a couple of seconds to read, you basically have to debug software, which takes a lot longer.
[+] [-] w4tson|11 years ago|reply
I think it's a straw man argument. An unfair comparison. Buildings & bridges etc have been designed and improved for literally thousands of years. Software is a half a century old.
Aza Raskin's blog piece about human powered flight which bubbled up on HN has always stuck with me[0]. "The problem is we don't understand the problem".
[0]: http://www.azarask.in/blog/post/the-wrong-problem/
[+] [-] spc476|11 years ago|reply
[+] [-] notacoward|11 years ago|reply
The confounding case is the Linux kernel, which is both large and systems-focused but seems to eschew both kinds of methodology in favor of undirected chatter on mailing lists. Then again, maybe that's just part of a two-level process. More than a few patches to the kernel have real specs behind them, though developers don't call them that. They're just "kind of long, kind of detailed" emails. This tendency to embrace a thing while rejecting the term for it is one of the weirdest tendencies many developers seem to have.
[+] [-] MBlume|11 years ago|reply
[+] [-] pjbrunet|11 years ago|reply
[+] [-] pjmlp|11 years ago|reply
[+] [-] alvatar|11 years ago|reply
You can apply the same metaphors to software ;)
[+] [-] ams6110|11 years ago|reply
Also from a practical standpoint you probably won't be able to get a building permit without blueprints or drawings of some sort (though this varies).
[+] [-] gadders|11 years ago|reply
[+] [-] WalterBright|11 years ago|reply
[+] [-] cfontes|11 years ago|reply
In software you most of the time have the 2 things running parallel because of time constraints and other business needs, which almost always leads to failure.
[+] [-] fsloth|11 years ago|reply
This probably one of the reasons some agile practitioners scoff at documentation - if one is using the same framework to implement the similar website time and again effectively the model remains the same and understanding of the model is easy to retain and retrain.
[+] [-] reitanqild|11 years ago|reply
> But few programmers write even a rough sketch of what their programs will do before they start coding.
deserves a reference to a study. (or, and this is the first time I use this word: a #strawman flag.)
- or am I the only one who sketches up data models and data flows before going to work?
Also: Prototyping comes early in many other designs as well, often way before detailed blueprints are signed off I think (not a mechanical engineer, just outside observations).
In software it just so happens that prototypes are often made by the same people and from the same material as the final product, only reinforced to withstand production load and abuse.
[+] [-] falcolas|11 years ago|reply
Anecdata: Yes, or rather you're one of a vast minority. I work in an office with no notebooks in the supply closet, and one 5x5 whiteboard which is usually full of UX layouts.
[+] [-] th3iedkid|11 years ago|reply
Proof systems have their uses and then there are places where they are just too much.The best is in the mid!
>> "am the creator of the TLA+ specification language." I think TLA+ is/was being used at Amazon [1]
Safety with Preservation and Progress guarantees of critical systems demand design audits at a larger scale however if the scale of the program is actually quite small , things like type-systems themselves are sufficient proof of safety.
Ofcourse there are cases where even typing is too much!
One of the middlepaths is actually using inspection tools on code/programs as another aspect altogether.
[1]:http://research.microsoft.com/en-us/um/people/lamport/tla/fo...
[+] [-] matt_s|11 years ago|reply
If it is a start-up with 5 people, the software probably is the specification. If it is a large organization of devs (25, 50 or 100 people), then you will see more documentation, some of that specifications.
There is likely a threshold in the number of people where the number of communication paths between them is too large to rely on informal ways of communication. Maybe its 15-25 developers. The larger an organization is the more bureaucracy gets introduced and you now have people whose role is to write specs and requirements docs, etc.
[+] [-] javindo|11 years ago|reply
[+] [-] hussong|11 years ago|reply
I would expect a house to last 100 years. A family member once bought a house that is over 400 years old (and still standing).
Then again, construction and houses are way more expensive in Europe/Germany. They're planning to tear down and rebuild my old high school from scratch, current estimate: 85 million euros total.
[+] [-] randartie|11 years ago|reply
[+] [-] kitd|11 years ago|reply
http://delivery.acm.org/10.1145/2740000/2736348/p38-lamport....
[+] [-] badpenny|11 years ago|reply
[+] [-] fsloth|11 years ago|reply
This.
[+] [-] collyw|11 years ago|reply
[+] [-] ori_b|11 years ago|reply
The process of producing code is the process of specifying what the computer should do -- the deliverable is the specification. On the other hand, few civil engineers will start designing plumbing without at least having a sketch of the floor plan and an idea of the building uses.
[+] [-] onion2k|11 years ago|reply
I'm sure coding is "specifying what the computer should do", but that's not what people are talking about when they talk about specifications. A specification defines what the application should do. Producing code is implementing the specification. Without having any specification first you wouldn't have a clue what the code needs to do, so you wouldn't know what to write.
The problem with the analogy is that most people don't understand what a blueprint actually is. A blueprint is an architectural planning document. It defines the building in very broad terms - where the walls go, where there's a staircase, and so on. It doesn't go in to the detail of how the building is constructed. That comes from all the other documentation. As an analogy with software it's a pretty good one - a building's blueprint is equivalent to the high-level planning documentation that any big project needs.
[+] [-] JoeAltmaier|11 years ago|reply
An experienced carpenter/contractor can build a house, without excruciating detailed notes. In fact they can build it faster, adapting to conditions as they change and solving problems efficiently.
[+] [-] herpelderp|11 years ago|reply
[+] [-] justincormack|11 years ago|reply
[1] https://en.wikipedia.org/wiki/Winchester_Mystery_House
[+] [-] lukesan|11 years ago|reply
[+] [-] erikb|11 years ago|reply
[+] [-] fsloth|11 years ago|reply
I would take ABNF as used in RFC:s and a few clarifying diagrams over verbose UML vomit any day :)
[+] [-] pjmlp|11 years ago|reply
For anything of scale we really need blueprints.