please stop linking this paper. it is too old, endlessly debated, and does a disservice to any precieved motive of promoting or demoting any of the mentioned languages.
offtopic: I've never posted any links on HN before, but is there no confirmation page upon submitting that tells you the link has been submitted before?
For each language, I would have two experts in that language. Call them P and D: programmer and documenter.
They would not be permitted to communicate.
P would be required to write strictly code, without a single comment or a shred of documentation.
D would not be permitted to view the problem description: only the input data (along with documentation describing what the data means) and the code written by P.
D would have to determine what the code is doing and specify the problem being solved, relying on his or her expertise in the implementation language to unravel how the code is treating the input data, and hence what is its meaning.
Then evaluate the time spent by P, the time spent by D, the size the code, and the degree to which the code actually solves the problem and the degree to which D's documentation actually reverse-engineers the specification not revealed to D.
Then we could say meaningful things: like a given solution is very small and was developed quickly, but it took great effort to unravel what it's doing. That's not bad: there are situations when that is acceptable, namely the development of one-off solutions that will never need to be understood or maintained, and which are needed ASAP. Situations would also reveal themselves when the program takes long, and isn't particularly short, but is a breeze for D to understand. Usually we would prefer that situation in permanent production code.
The all round winner would be: small code written by P in a short amount of time, quickly reverse-engineered by D to recover the spec. (With bonus points for time performance, small memory use, etc).
There is also a degree of realism in this setup. How often are we called upon to understand some code which is utterly undocumented? That's when we feel the differences in languages, programming approaches and styles.
Tradeoffs in speed vs maintainability could be the intentional result of choices made by P though. What instructions would they be given regarding how to handle those tradeoffs?
Hm, this paper is pretty obviously promoting Haskell. The main authors wrote the haskell variant, did not even execute the other prototypes and make conclusions without any sort of comparison except LOC.
I like haskell, but this paper is doing a poor job of representing scientific honesty.
The people conducting the test also, without the authors' knowledge, got a grad student, gave him eight days to learn Haskell (with no formal training, but the option to ask a more experienced Haskell programmer questions), and then handed him the problem spec and told him to implement it in Haskell. He did it in eight hours with a bit under twice as many LOC as the paper's authors' version. The paper's authors also point out issues with the study, and describe it as showing the possible worth of functional languages, not Haskell specifically.
They added many disclaimers saying that LOC alone is a bad metric and it's more interesting to look at characteristics of the code for anecdotes. The article is full of of advice to take the findings with a grain of salt.
They don't even mention haskell in the conclusion, they just say that functional programming is well-suited to rapid prototyping.
Someone (I think John C. Peterson) mentioned this paper at the Paul Hudak memorial symposium last year. His comment was the that the comparison was unfair, because the Haskell program was written by Mark P. Jones, who was a good enough programmer to outshine his competitors no matter what language. :)
The second Haskell program in this paper, that also out compete everyone, was built in a week by a grad student that did not knew Haskell before hand without talking to the first implementer.
I do not think that someone wants to use C++11/14 to make app prototype. There are a lot more languages now which in my opinion are better suited to prototyping than languages mentioned in article. Most of people will use Python, JS or some other scripting language.
There might be many biases in this research. If one is able to write code in Haskell it tells a lot about them, and he might be a better programmer than one who writes in Ada or C++.
To make this research unbiased we need to do the following:
* Take people who are profecient in Ada, Haskell and Ada
Seasoned and experienced Ada programmers are likely to come from a high-level (software) engineering background in some large company, e.g. in the aviation industry or military. Haskell programmers will more likely work in academia, e.g. a professor of computer science. And the C++ programmers could come from any field or industry.
These people will likely shine with certain tasks and be less productive with other tasks, depending on their varying skill sets. So you're probably only measuring the overall level of proficiency of programmers of certain languages in a particular programming domain. For example, I'd bet that the average experienced C++ programmer is slightly less skilled than the average experienced Ada programmer (if both of them are mostly working with their language professionally), simply because C++ is more widespread whereas the few Ada jobs left require a relatively high level of expertise. On the other hand, if you're writing programs in Ada for Airbus, you're probably accustomed to a slower development pace than if you write C++ code for a small gaming company, so the "productivity" will appear to be fairly low in comparison.
Then there are measures like LOC that are fairly meaningless across languages and perhaps even in general.
Finally, languages are usually chosen for their tool and library support and on the basis of available developers anyway.
"There might be many biases in this research. If one is able to write code in Haskell it tells a lot about them, and he might be a better programmer than one who writes in Ada or C++."
I'm going to bite as an Ada fan to say this is probably true to a degree since it's a high-level, functional language. Haskell similarly gets slammed in productivity by Common LISP which is high-level, less safe, has fast iterations, and good tools. However, if we're talking apples to oranges, Haskell already failed at doing the kind of low-level, efficient, and/or real-time apps that Ada was designed for vs Ada that makes it straightforward. The House operating system was a celebrated achievement and embedded use Haskell-based DSL's extracted to C for good reasons. Meanwhile, SPARK variant is allowing proof of absence of key errors without people being experts in theorem provers and with most of the efficiency of Ada vs provable ML/Haskell.
Yeah, I'd love to see a bunch of teams trying to write deterministic, safe, and fast software in each of these languages. Let's see about protocol engines, string libraries, real-time apps, high-performance on single cores, lightweight containers, provable absence of problems in component, and so on. I'm betting on Ada 2012/SPARK 2014, C++, and Haskell in that order given attempts to balance all those requirements.
I wonder if there's ever been even a single comparative programming languages study which would adhere to the standards routinely required in medical and epidemiological studies?
> If one is able to write code in Haskell it tells a lot about them, and he might be a better programmer than one who writes in Ada or C++.
And here's the arrogant, self-important attitude that turned me off of Haskell.
When I was using it, I didn't see any evidence that Haskell users are any better or worse than programmers in any other language.
If it were true, why aren't Haskell developers taking over the world with their superior software?
In reality, any new, interesting language is going to attract better than average developers at first, because those developers are always learning new stuff and trying different things.
That is insanely low. Odd to see that it gets basically a passing mention. They acknowledge the absurdly fast development time. Gave the language an "on par" evaluation (though, how they account for a C in prototyping support in what was hands down the winner here is, interesting), and then move on.
For those that didn't click through, it was about half the time of the next fastest solution.
Edit: I am a really forgetful person. Seems I've made this same observation on this same paper every time it comes up. My apologies if the same people are reading it. :)
BTW... in the diagram showing the regions of interest for a given set of craft and their positions, some of them are referred to as "doctrines". Anyone know what "doctrine" means in this context?
[+] [-] platz|8 years ago|reply
previous discussion:
https://news.ycombinator.com/item?id=13275288
https://news.ycombinator.com/item?id=7050892
https://news.ycombinator.com/item?id=7029783
https://www.reddit.com/r/programming/comments/1v0z02/haskell...
[+] [-] partycoder|8 years ago|reply
[+] [-] syockit|8 years ago|reply
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] kazinator|8 years ago|reply
For each language, I would have two experts in that language. Call them P and D: programmer and documenter.
They would not be permitted to communicate.
P would be required to write strictly code, without a single comment or a shred of documentation.
D would not be permitted to view the problem description: only the input data (along with documentation describing what the data means) and the code written by P.
D would have to determine what the code is doing and specify the problem being solved, relying on his or her expertise in the implementation language to unravel how the code is treating the input data, and hence what is its meaning.
Then evaluate the time spent by P, the time spent by D, the size the code, and the degree to which the code actually solves the problem and the degree to which D's documentation actually reverse-engineers the specification not revealed to D.
Then we could say meaningful things: like a given solution is very small and was developed quickly, but it took great effort to unravel what it's doing. That's not bad: there are situations when that is acceptable, namely the development of one-off solutions that will never need to be understood or maintained, and which are needed ASAP. Situations would also reveal themselves when the program takes long, and isn't particularly short, but is a breeze for D to understand. Usually we would prefer that situation in permanent production code.
The all round winner would be: small code written by P in a short amount of time, quickly reverse-engineered by D to recover the spec. (With bonus points for time performance, small memory use, etc).
There is also a degree of realism in this setup. How often are we called upon to understand some code which is utterly undocumented? That's when we feel the differences in languages, programming approaches and styles.
[+] [-] aninhumer|8 years ago|reply
[+] [-] ysleepy|8 years ago|reply
I like haskell, but this paper is doing a poor job of representing scientific honesty.
[+] [-] jejones3141|8 years ago|reply
[+] [-] emef|8 years ago|reply
They don't even mention haskell in the conclusion, they just say that functional programming is well-suited to rapid prototyping.
[+] [-] vilhelm_s|8 years ago|reply
[+] [-] di4na|8 years ago|reply
See this for a pretty powerful rebuttal.
The second Haskell program in this paper, that also out compete everyone, was built in a week by a grad student that did not knew Haskell before hand without talking to the first implementer.
[+] [-] gertef|8 years ago|reply
Ah :-)
> memorial symposium
Oh no :-(
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] acadien|8 years ago|reply
[+] [-] digitalzombie|8 years ago|reply
Likewise C++ isn't going to be as concise as Haskell ever.
So the paper still hold some truth if not mostly valid.
[+] [-] marcosdumay|8 years ago|reply
Still, they are so different from each other that I don't think all that change makes much of a difference.
[+] [-] flukus|8 years ago|reply
[+] [-] oselhn|8 years ago|reply
[+] [-] solomatov|8 years ago|reply
To make this research unbiased we need to do the following:
* Take people who are profecient in Ada, Haskell and Ada
* Randomly assign them to different teams
* Compare their results
[+] [-] jonathanstrange|8 years ago|reply
Seasoned and experienced Ada programmers are likely to come from a high-level (software) engineering background in some large company, e.g. in the aviation industry or military. Haskell programmers will more likely work in academia, e.g. a professor of computer science. And the C++ programmers could come from any field or industry.
These people will likely shine with certain tasks and be less productive with other tasks, depending on their varying skill sets. So you're probably only measuring the overall level of proficiency of programmers of certain languages in a particular programming domain. For example, I'd bet that the average experienced C++ programmer is slightly less skilled than the average experienced Ada programmer (if both of them are mostly working with their language professionally), simply because C++ is more widespread whereas the few Ada jobs left require a relatively high level of expertise. On the other hand, if you're writing programs in Ada for Airbus, you're probably accustomed to a slower development pace than if you write C++ code for a small gaming company, so the "productivity" will appear to be fairly low in comparison.
Then there are measures like LOC that are fairly meaningless across languages and perhaps even in general.
Finally, languages are usually chosen for their tool and library support and on the basis of available developers anyway.
[+] [-] nickpsecurity|8 years ago|reply
I'm going to bite as an Ada fan to say this is probably true to a degree since it's a high-level, functional language. Haskell similarly gets slammed in productivity by Common LISP which is high-level, less safe, has fast iterations, and good tools. However, if we're talking apples to oranges, Haskell already failed at doing the kind of low-level, efficient, and/or real-time apps that Ada was designed for vs Ada that makes it straightforward. The House operating system was a celebrated achievement and embedded use Haskell-based DSL's extracted to C for good reasons. Meanwhile, SPARK variant is allowing proof of absence of key errors without people being experts in theorem provers and with most of the efficiency of Ada vs provable ML/Haskell.
Yeah, I'd love to see a bunch of teams trying to write deterministic, safe, and fast software in each of these languages. Let's see about protocol engines, string libraries, real-time apps, high-performance on single cores, lightweight containers, provable absence of problems in component, and so on. I'm betting on Ada 2012/SPARK 2014, C++, and Haskell in that order given attempts to balance all those requirements.
[+] [-] sampo|8 years ago|reply
I wonder if there's ever been even a single comparative programming languages study which would adhere to the standards routinely required in medical and epidemiological studies?
[+] [-] jlarocco|8 years ago|reply
And here's the arrogant, self-important attitude that turned me off of Haskell.
When I was using it, I didn't see any evidence that Haskell users are any better or worse than programmers in any other language.
If it were true, why aren't Haskell developers taking over the world with their superior software?
In reality, any new, interesting language is going to attract better than average developers at first, because those developers are always learning new stuff and trying different things.
[+] [-] bwackwat|8 years ago|reply
Anyone have a modern example of this type of research? Opinions are welcome!
[+] [-] lispm|8 years ago|reply
[+] [-] taeric|8 years ago|reply
For those that didn't click through, it was about half the time of the next fastest solution.
Edit: I am a really forgetful person. Seems I've made this same observation on this same paper every time it comes up. My apologies if the same people are reading it. :)
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] marcosdumay|8 years ago|reply
I'm sure it could become more readable with some more time invested.
[+] [-] jejones3141|8 years ago|reply
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] arnon|8 years ago|reply
[+] [-] solomatov|8 years ago|reply
[deleted]