top | item 10333722

The economics of software correctness

113 points| exupero | 10 years ago |drmaciver.com | reply

53 comments

order
[+] brightball|10 years ago|reply
Everything in development is a cost-benefit trade off. Despite it's speed limitations, Ruby remains an excellent language for development because of its focus on developer efficiency. You can produce feature complete systems in a short period of time at the expense of slightly higher hosting costs, which is significantly cheaper than labor costs.

Monolith development is still much faster than microservice development even though microservice development is the better long term option by far.

It's all tradeoffs. I'm an architecture nut and for years I really wanted nothing more than to design the ultimate perfectly scalable and secure system but unless you're at a virtually competition free enterprise, like a telecom or an insurance company the time or budget to do so probably doesn't exist.

I've gone from "get it done" to "do it perfect" back to "get it done and avoid obvious problem starters". The reality is just that "get it done" wins the business case almost every single time.

[+] mreiland|10 years ago|reply
I agree within reason. I believe there are certain things that must happen, even with a 'get it done' mentality or the subsequent gitrdone is not possible due to productivity issues.

Things like error detection and reporting in the system absolutely have to be useful. I've seen systems where that isn't the case, and it destroys productivity.

So I agree in general, but I think there are a few areas that you need to get right or you can't gitrdone effectively for very long.

[+] kybernetikos|10 years ago|reply
So this is all true, however it is surprising how often in development you find people spending more time and effort getting the 'pragmatic' hack to work acceptably than it would have taken to do it right.

People often underestimate the difficulties of getting a hack to work and overestimate how long doing something correctly will take. Even with short time horizons.

[+] eCa|10 years ago|reply
> People often underestimate the difficulties of getting a hack to work and overestimate how long doing something correctly will take.

Yes. And sometimes one of those people is your boss.

[+] mreiland|10 years ago|reply
I personally hate working with people who put things in terms of 'right' and 'wrong', or 'correct' and 'incorrect'.

I honestly believe the key to software development is good decision making. If you're thinking in terms of right and wrong, correct and incorrect, then you're not making decisions, you're simply doing the right thing over and over again. Only it may not be the OPTIMAL thing, or the smart thing, in any specific instance.

For example, who here really believes proponents of unit tests have never painted themselves into a corner? How did that happen? By making poor decisions.

Stop thinking in terms of right and wrong. Seriously. Sometimes unit tests are a good decision, sometimes they're not. Sometimes that hack is a good decision, and sometimes it isn't.

You make a decision on a case by case basis, not because it's right.

[+] davidw|10 years ago|reply
"The problem is not that we don’t know how to write correct software. The problem is that correct software is too expensive."

That's something I've been saying for a while. It's all about the economics of the situation, rather than the impossibility if it.

Naturally, there are also improvements here and there that decrease bugs without increasing efforts, and those are worth looking for.

[+] shasta|10 years ago|reply
But in a practical sense, the problem is that we don't know how to write correct software.
[+] zappo2938|10 years ago|reply
Joel Spolsky published a blog post in 2000 named "Things You Should Never Do" (http://www.joelonsoftware.com/articles/fog0000000069.html)

It's about why Microsoft with IE6 won the browser war against Netscape who made the single worst strategic mistake a software company can make by rewriting Netscape 6.0 throwing out all the code from Netscape 4.

Netscape was working with extremely buggy and convoluted code in the older version and trying to save the development community from the nightmare that is IE6, ended being late to the market with a superior product. Joel makes a very good observation that often people want to throw out old code because they think it's a mess, but the truth is counter intuitive that the old code contains vast amounts of knowledge.

A company can be first, best, or cheat and in this case while Netscape was trying to be best, Microsoft was first.

This is the reason iterative code development is best. Speed of iteration beats quality of iteration 9 out of 10 times. Boyd's Law of Iteration (http://blog.codinghorror.com/boyds-law-of-iteration/) The best software is software that released most often, not released the most correct.

If I was to release a browser in say 2008 to compete with the dominance of IE, what is the single most important feature I could put into that browser? I'd put a feature for forcing iteration, so that the browser can automatically update on the client finishing a development cycle rather than release the update preinstalled unable to remove on newly bought computers.

[+] danjayh|10 years ago|reply
Funny thing is, Microsoft did almost the same thing with Windows NT. However, unlike Netscape, they had the resources to keep iterating their old junk while they worked on the totally new version.
[+] jaggederest|10 years ago|reply
I think one of the other problems is that people value the economics of software correctness using their gut, rather than empirical analysis.

https://en.wikipedia.org/wiki/Hyperbolic_discounting

Everyone knows that bugs are problematic eventually, it just seems that they can't put that on a level playing field with the up front costs, be they in terms of time, features, or effectiveness.

As an example, if you asked Home Depot whether they were saving money with their self checkout machines, I'm sure the answer would be different before their data breach vs after. Even after being warned they simply couldn't properly discount the possibility of massive damage in the future when offered a short term benefit.

[+] nostrademons|10 years ago|reply
Hyperbolic discounting is rational when the availability of resources increases exponentially, as it often does for products that catch on.

A company that comes to market with a product that is useful but buggy will attract the attention of venture capitalists & other investors. It will receive user feedback from its existing user base. It will find it easier to hire top talent. It will be able to use collected data to make better products. All of these factors are in proportion to the company's size, which tends to make growth rates exponential.

It's pretty standard practice in the tech industry to bring a buggy, barely-working product to market; use interest in that to raise money; use money to hire engineers; and use the engineers to fix the bugs. You could even look at this as a net benefit to society, as long as existing customers would rather use the product in its buggy, incomplete state than go without it.

[+] methodover|10 years ago|reply
> You have probably never written a significant piece of correct software.

So true.

When we were hiring for a senior engineer position a few months ago, one applicant said he wrote "bug free code."

I laughed, sent it around on slack. We all chuckled at it. The applicant did not get a call.

[+] wellpast|10 years ago|reply
> If you want better software, make or find tools that reduce the effort of finding bugs. ... Better monitoring is another. Code review processes. Static analysis. Improved communication. There are many more.

I'm going to come out and suggest that even THESE are more expensive than most businesses need. I believe code review processes are insanely expensive for what they usually return (wrt bug costs).

[+] draw_down|10 years ago|reply
Main takeaway: "Buggy software is not a moral failing."
[+] hyperpallium|10 years ago|reply
What about reducing bugs through code reuse, in the form of libraries and frameworks? Because they have more users, more bugs have been found, reported and fixed. e.g. using standard libraries instead of writing your own.
[+] swalsh|10 years ago|reply
code reuse is actually dangerous though. It's possible to use a library, have an expectation on how it will work, and then forget about that assumption, upgrade/fix something in the library, and proceed to break your assumption. It's a pretty common problem.

Often the time saved using a library is worth the small risk, but if we're talking about "how to write correct software" you'll want to be weary of code reuse.

[+] InclinedPlane|10 years ago|reply
That works only when the quality of the libraries and frameworks are high. When they are crap, you just spread the pain around and you've made things worse.
[+] DRMacIver|10 years ago|reply
Most libraries found in the wild are kinda crap, because as an industry we have a massive problem where most of our stuff is built on free labour that people have performed in their spare time to scratch their own itch.

Using libraries is still generally a good idea, but its effect on software correctness is a bit of a coin toss.

[+] jordanchan|10 years ago|reply
Reading this made me wonder...do SpaceX and other private space flight programs have similar standards of rigor?
[+] Joeri|10 years ago|reply
Probably not, which is why I expect spacex to put a person on Mars long before NASA does.
[+] a3voices|10 years ago|reply
It's possible to write software with very few bugs if you're very experienced in the language/framework and software development in general, writing all the code yourself, and perfectly understand the requirements. Otherwise, good luck.
[+] amelius|10 years ago|reply
Sometimes, writing bug-free code isn't even sufficient; sometimes, one needs proof that it is bug-free.

Then, your favorite language/framework will not help you much. You'll need strong verification tools, and a language and frameworks that are compatible with it.

[+] AnimalMuppet|10 years ago|reply
It's even better if you're working with others who all are "very experienced in the language/framework and software development in general, and perfectly understand the requirements" - at least if you all communicate well with each other. Code reviews and people to bounce ideas off of reduce bugs, they don't add them.