Ask HN: Is there evidence regarding PG's theory about powerful languages?

[kartickv]

Paul Graham claims that startups should use the most powerful language for a job, like Lisp. I'm looking for statistical evidence for or against this theory.

For example, amongst the thousands of YC startups, have ones that used Lisp or Smalltalk or other esoteric but powerful language succeeded more than startups that used a mainstream language like Python, Java or C++? Do we have statistics of success rate by language?

One should not look at a single startup (ITA software) that used Lisp and succeeded and conclude that it pays for the average startup to use Lisp (or other powerful language like Smalltalk).

[hiphipjorge]

This is totally subjective, but I think we're at a point where the language ecosystem is much more important than the actual language. This was probably not the case when pg was writing software for Viaweb and there wasn't nearly as much FOSS that you could just use. At that point in time the LANGUAGE was probably much more important because there was not a lot of differentiation beyond that. Now, we're at a point where the language isn't as important and the ECOSYSTEM around it is much more powerful that then language itself.

[hga]

That's certainly true. While Clojure isn't exactly my favorite Lisp (it isn't even a LISP, as in a LISt Processing language any more, and is rough around the edges in various places), running on the JVM ecosystem makes it a great choice for many things (well, aside from logging, standard JVM logging is "sacrifice a chicken" territory, but there are native solutions now as well).

Want to us a RDBMS? There are many libraries for doing that, my favorite is Korma, and they all use the JDBC interface and battle tested JDBC implementations, that part of the plumbing is trivial. Etc.

[a_bonobo]

I've observed this in bioinformatics -

Python, Perl, R are much more common than (let's say) lisp or Ruby, partially because more people know these first languages (possible exception in R - is that even taught in university?), but IMHO more because libraries such as BioPerl, Biopython, Matplotlib (Seaborn), numpy/scipy, ggplot2, the entire CRAN ecosystem are so much more mature than what you get in lisp/Ruby, you can get amazing stuff done very quickly.

[collyw]

I was thinking something similar. I used to use Perl until around 5 years ago, and its a pretty expressive language, great for building all the blocks for your application quickly. Then I discovered Django, and all those blocks I would have been building have been built for me.

[Kinnard]

Certainly javascript via npm is the best example. Javascript is eating everything because of its massive ecosystem.

[paulsutter]

Paul Graham says that Viaweb was able to run circles around their competitors because Lisp gave them a big advantage[1]. Of course, Facebook ran circles around their competitors. Myspace, friendster, left in the dust.

So is PHP more powerful than Lisp? (/s) His logic suggests so (given the relative performance of Viaweb vs Facebook).

Having great programmers matters more than using any specific tool.

[1] from http://www.paulgraham.com/avg.html

"It must have seemed to our competitors that we had some kind of secret weapon-- that we were decoding their Enigma traffic or something. In fact we did have a secret weapon, but it was simpler than they realized. No one was leaking news of their features to us. We were just able to develop software faster than anyone thought possible."

EDIT: Added (/s) since sarcasm was apparently nonobvious

[crdb]

I'll paraphrase Facebook themselves [1] back at the statement of yours that I disagree with ("His logic suggests so"): (A => B) !=> (B => A).

That Viaweb was competitive because of blub, does not imply that blub is a necessary condition for any company to be competitive, or that not-blub would impede Facebook from becoming successful.

For example, if fast new features are only possible with blub, and new features are not as essential to Facebook's success as they were to Viaweb's, Facebook's use of not-blub (or perhaps, non-use of blub) would not materially impact Facebook's success.

One way in which this might happen is: Viaweb is B2B, and its customers value new features; whilst Facebook is B2C and a social network i.e. the value is in the number of people on the platform, not its new features.

Another explanation might actually just be that there was no high enough level language in Viaweb's days to compete with Lispers, and plenty in the PHP era. To an extent we witness this today with the myriad frameworks available which in theory eliminate any edge for fast prototyping except in safety-critical or performance-critical applications.

[1] https://facebook.github.io/react/blog/2016/01/08/A-implies-B...

[erik14th]

Different times, different problems, apples, oranges.

I agree with pg that lisp was probably the best tool he had available at the time for his specific problem, ViaWeb was a system that generated systems, sounds like a lispy problem. And it also worth of note that not all of it was in lisp, so they mostly used languages that were good at each specific problem.

If your problem involves shipping quickly and iterating, php, the platform is very powerful, especially at the time facebook started.

So PHP may have been facebook's lisp, remember, rails and django were released on 2005, facebook started on 2004. So was php a bad choice? I don't think so. It might have been the best choice at the time.

What you should take from pg's essays on lisp is not that lisp is the holy grail of programming, but that picking the best tool for the problem may play a big part on your success.

Think about it, erlang is just perfect for whatsapp, so is lisp for ViaWeb's metaprogramming needs, and I think facebook didn't really have any big, complex problems at first, so picking the simplest, quickest platform to develop on was probably the best bet they could have made.

So, yes, PHP was more powerful, or better put, more empowering than lisp for facebook's specific problem.

[hga]

Of course, Facebook ran circles around their competitors. Myspace, friendster, left in the dust.

Friendster infamously was left in the dust because its VC laden board was more interested in striking cool deals than allocating the resources necessary to make the site fast enough, so that e.g. simply logging in didn't take a minute or more.

They, not a founder, were calling the shots, and while I don't know the Myspace story, I note they were bought by the monster sized News Corporation a year and a half after Facebook was founded, and during the period where their relative market positions flipped, so blame it on Rupert Murdoch if you want ^_^, but the chances they had nimble management after the acquisition are very very low.

Getting back to Friendster, as a DEC published history of microcomputer companies in the '70s put it, the ones who survived at minimum did an adequate job of everything essential, from all the different bits of hardware (no one ever bought DEC for the quality/price-performance of their disk drives), to documentation, to support etc. So no matter how powerful your language is, if you don't, for example, run it on enough servers, or architect your system so that's even possible, you're not going to succeed.

[wasdfan]

You're engaging in base rate neglect: the pool of PHP programmers was vastly larger than the pool of LISP programmers when Facebook was founded.

[bcjordan]

Aaron Swartz wrote in 2005 about Reddit's switch from Lisp to Python, noting some upset reactions in the Lisp community.

http://www.aaronsw.com/weblog/rewritingreddit

> Another figured something else must be going on: “Could this be…a lie? To throw off competition? It’s not as though Paul Graham hasn’t hinted at this tactic in his essays…”

[striking]

I can't give you statistical evidence, but it's well known (or often said, at least) that using the right tool for the job makes the odds of success a lot higher.

That being said, "right" is very different from pg's "powerful", especially as he says:

> Lisp is so great not because of some magic quality visible only to devotees, but because it is simply the most powerful language available.

In my opinion (and this person's[1]), Lisp is "powerful" in that it lets you express anything quickly and easily in a way that makes sense to you.

That doesn't mean it makes sense to anyone, nor does that mean it's compatible with anything else.

And where he says:

> What's less often understood is that there is a more general principle here: that if you have a choice of several languages, it is, all other things being equal, a mistake to program in anything but the most powerful one.

I feel he is sorely mistaken. The best programming language for the job is the one that most easily and quickly fits the requirements, given your experience level and the intrinsic attributes of the languages in question. That may be less true when you're working on a personal project rather than one for work... but from an engineering standpoint, development speed is very important.

PHP may be near-universally reviled as a language, but I won't contest the fact that it runs anywhere, quickly. If you needed to quickly put up a blog at least some time ago, the answer was almost always "use Wordpress and PHP".

Python is slow. Painfully slow. Slow as in "it stores the entire AST at runtime and lets you modify it while the program runs" slow. But it's also great at numerical computation with numpy. And it's enormously flexible (which is what makes it slow). And it's clean. It writes really well. For a quick mathematical processing program or website whose code needs to be legible, it's great.

This goes on for just about any language. There are reasons to, and not to, use a programming language. It's not just "power" that's important, it's a number of factors. I hate many languages with a passion, but I'll agree that those languages still have their uses. So I disagree with pg here.

1: http://www.lambdassociates.org/blog/bipolar.htm

[mr_luc]

> That being said, "right" is very different from pg's "powerful"... > Lisp is "powerful" in that it lets you express > anything quickly and easily in a way that makes sense > to you. That doesn't mean it makes sense to anyone ...

Graham and his cofounder (and later Trevor Blackwell) all wrote different areas of the product, in different languages.

Joe Armstrong (creator of Joe's Abstract Machine) has also argued very recently for this kind of development.

I'll be the first to admit that I haven't seen it work myself. But when the people practicing it are at that level, maybe the problem is just that there aren't enough really good programmers?

[EuAndreh]

I believe that is what he meant by "all other things being equal". They are equal in library availability, speed, etc.

[DasIch]

Python doesn't store the entire AST at runtime. You can hook into the import process and modify code before it's compiled to bytecode. You can modify the bytecode at runtime but not any that's on the stack.

Python is also not all that slow, CPython is slow-ish but it's has effectively no optimizations at all. PyPy which has decent optimizations is actually fairly fast and getting faster with every release.

[Terr_]

From a business/ERP-ish standpoint, I believe one of the most important aspects of a language (or framework) is that you minimize how many "translation steps" exist between how the business sees the world and how the programmer is able to implement it.

[tyre]

No, but you can frame it in a way where he was correct.

Look at some of the largest tech companies and what languages they used to get big:

  - Facebook: PHP
  - Twitter: Ruby
  - Dropbox: Python
  - Box: PHP
  - Github: Ruby
  - AirBnB: Ruby

Probably not what PG would have had in mind.

If you move away from his objective claim about Lisp's "power" to a subjective metric based on "the right tool for the job", then sure. (Though reading e.g. Hackers and Painters, his argument around Lisp was specific to the power of that language.)

PHP perfectly fits the "move fast and break things" mantra of early Facebook. Ruby allowed for quick iteration at early Twitter.

Note that Facebook built a PHP-to-C++ compiler called HipHop, while Twitter switched largely to Scala. Box (to the best of my knowledge) has supplemented PHP with Java and Scala. Github maintained a custom fork of Rails for a long time to achieve the performance it needed.

So no, it doesn't appear that Lisp-flavoured companies have an advantage.

[incepted]

All good points.

Even though it's hard to find any flaw in "Use the right tool for the job", I still think that the success of a start up in Silicon Valley (and probably the rest of the world) has a lot less to do with the technical choices of the company than the people who work there. And luck. A lot of luck.

It's still baffling to me that pg doesn't realize how lucky he was. He was in the right place at the right time and he was probably surrounded by competent technical and non technical people. If you have such a team, the technical preferences of the engineers will probably have very little bearing on the success of the company as long as they use the tools they are proficient with and which allow them to execute quickly.

I mean, the fact that Rails and PHP were at the origin of Twitter and Facebook alone shows that the quality of programming languages is irrelevant for the early year(s) of a start up.

[taneq]

It's true that you can frame it that way, but I think it's clear that this wasn't the spirit of what he was saying, and is certainly a far less controversial message ("use the right tool for the job" is hardly groundbreaking!)

I always thought he was speaking largely from survivor bias - "we succeeded, and we did X, therefore X must have contributed to our success."

[wasdfan]

This is base rate neglect: the percentage of people who are using elite tier languages (LISP, etc) are tiny in comparison to the number of people slinging Ruby/Python/etc. All that needs to be shown is that, per capita, LISP users tend to do better. Unfortunately, I don't have such data at my fingertips :)

[kartickv]

This is very useful, thanks. I'll concentrate on learning Ruby.

[nl]

Read the date on those essays. You'll note that they were all written prior to YC building momentum.

I think it's fair to say they are based on PG's early experiences, but it didn't appear to generalize.

You'll note he stopped writing about it, and YC funded plenty of companies that used many, many different languages.

[wasdfan]

This is base rate neglect.

The percentage of people who are competent enough (or crazy enough) to know a LISP well enough to build a product out of it AND be interested in a startup AND applying to YC is quite small in comparison to the number of people slinging code in the more popular languages (who are also interested in applying to YC).

Of course YC is funding companies that don't use i.e. LISP/Haskell/whatever is cool. If it only limited itself to the elite/hipster programmer startups (heh), it wouldn't have that many companies to invest in per year.

[brudgers]

This was more or less my take. When Viaweb was built in the mid 1990's, Python was not a mainstream language, Java was new, and building web sites in C++ was approximately as efficient as it is today. The scale of the web was different in that Viaweb had a server in their office. It was also the server.

By the first batch of YC, Graham had written about Python as an example of a leading edge language that was a good way to filter for "better programmers". Not long after the first batch started, Reddit had ditched Common Lisp for it.

Anecdote is not the singular of data, but Whatsapp choosing Erlang is a more recent example of using the right tool for the right job.

[wasdfan]

I can only speak from anecdotal memory, but I'm pretty sure Wit.AI primarily used clojure (arguably one of the most expressive lisps of all time) to go from 0 to acquisition in 18(!) months for what was essentially a pure software product.

[stray]

You will find no meaningful statistical evidence because it can't be done -- programming languages don't write software. Programmers do -- and programmers are made of meat.

A bad programmer using Lisp will be less successful than a good programmer using PHP.

And startups almost universally (regardless of the back-justifications they come up with) choose programming languages based on fashion: Résumé driven development.

The successful programming language for a startup is one that the product can be built in -- and that investors will not be alarmed by. PHP would not be alarming at all to many investors -- so PHP is a decent choice if you intend to sell your company off anyway.

[dvirsky]

Usually it's just one of the languages the technical founder is most familiar with. Mark Zuckerberg's weapon of choice before Facebook was PHP - which was one of the best options for web dev circa 2004 - so he just used the most productive tool for him.

[alayne]

Alan Kay has a more generalized idea about building up abstractions that I like better. I think he mentioned it in this video https://www.youtube.com/watch?v=NdSD07U5uBs

I, like many in this thread, think that good people are more important than a particular programming language.

[jandrewrogers]

Note that he says the most powerful language for the job. Which language is the "most powerful" varies greatly by application, and would include mainstream languages in a variety of contexts. For example, C++11 is the power language of choice for databases engines, and a reasonable argument could be made that languages like Python and PHP are as well in certain contexts. On the other hand, some application domains are arguably better served with esoteric languages.

In other words, use the best tool for the job. LLVM IR also deserves mention for its role as a kind of ur-language that can express all others in many modern applications. The ability to easily design and dynamically compile purpose-built languages is proving to be useful in many contexts.

[agumonkey]

There may be a little more structure behind this. Lisp, at his time, was incredibly different and prone to creativity and abstractions[1]. Amazing vehicle to explore. Most of the socially successful languages aren't about exploration but about economy of scale, lever. Most people are more happy building application by standing on shoulders of libs rather than finding new solutions to odd problems. I feel it's a bit similar to Haskell these days. They reinvent a world with their own abstractions, extremely powerful ones (concise yet general yet fast) but lots of people don't want to abstract this way.

[1] at a cost, you'd need high end machines to enjoy the benefits without the memory/gc issues.

[stray]

Lisp (Common Lisp, anyway) has not changed in this century -- and even 20 years ago, Common Lisp didn't require a very high end machine (at work I had a modest Sun workstation and at home a Macintosh -- and both did just fine with Common Lisp).

It's still an amazing vehicle for exploration -- and it's still very good for building software. Some of Lisp's benefits lost their impressiveness because other languages have included those things as well.

But we live in BlubWorld.

And most people don't see the value in languages that look weird to them. They can't.

[a-saleh]

I found a blog-post that tries to do this sort of comparison: http://codingvc.com/which-technologies-do-startups-use-an-ex...

I think there are two valid reasons for choosing non-mainstream language.

First, you know the language and want to work in it. A.f.a.i.k that was the main reason why PG chose it to build his company.

Second, if you know the technology is a good fit for your product, for example WhatsApp and Erlang seem to be good example for this.

[d0m]

I think you should use whatever you're best with. There are so many risks in a startup that you might as well not add another technical risk. PG was a lisp expert so it made sense for him to use it.

[kristianp]

Interesting that no-one has pointed out here the dynamic vs static or scripted vs compiled dimensions.

Facebook's use of php arguably allows them to build new features quickly, just as Viaweb's use of Lisp. But is it the dynamic nature of those languages that allows the faster development, not the 'power'?

It seems accepted that dynamic languages allow faster development, but static languages are more able to prevent bugs. It varies from language to language of course.

Apparently the research studies on static vs dynamic have many problems though, so you can argue that there is no evidence that one is "better" than the other: http://danluu.com/empirical-pl/

Maybe we should have a coding competition that has an equal number of (lisp, perl, php, ruby, python) experts vs (haskell, ocaml) experts and see how fast they can add features, vs how buggy those features are. Haskell and Ocaml are powerful, but 'very static', so it would be an interesting comparison.

[ankurdhama]

There are enormous number of factors that play a role in the success of startups and most of them are more important than the programming language they use. By the way I have never understood how "powerful" can be the property of a programming language? If you have people with good experience in X language then no matter how "powerful" language Y is, it cannot compete with that experience.

[wasdfan]

Consider Assembly as a counter-example.

[lpolovets]

This is somewhere between anecdotal and statistical evidence, but I wrote a post about technology vs. startup quality a few years ago: http://codingvc.com/which-technologies-do-startups-use-an-ex...

The gist of the post was to correlate technologies listed on companies' AngelList profiles with those companies' Signal scores. Signal scores are a bit of a black box, but I was told by an AngelList employee that they're roughly like PageRank over the graph of startups, founders, and investors. That is, a startup's Signal score will be higher if the people affiliated with that startup have high Signal scores.

There were some correlations between higher Signal scores and lower usage of PHP, higher usage of languages like Go and Scala, etc. That supports PG's point, but probably lacks the statistical rigor that you'd want.

[kartickv]

This is brilliant. What I was looking for. Thanks very much. It seems that great startups use JS, Ruby, Python and Java, in that order.

[mbrodersen]

The ecosystem (libraries, documentation, tutorials etc.) are much more important than the language. AND the biz ideas/execution are MUCH more important than the programming language ecosystem. Programming languages doesn't build successful businesses (unless you are a programming language business).

[tim333]

It's hard to figure statistics for the stuff as it's all a bit vaguely defined which is kind of inherent to the nature of the problem. In the essay http://www.paulgraham.com/power.html PG suggests power can be equated with susinctness.

I'd kind of guess the most succinct language for startups to write in would tend to be Ruby as there is so much existing code to do things - Rails, gems and all that. It also seems associated with successful startups like Twitter, Github etc so maybe there's a correlation there?

(Disclaimer - I'm not a Rails programmer - kinda guessing there).

[AnimalMuppet]

I'll give you an argument that PG's theory (at least as stated) cannot be correct.

PG says that Lisp is at the top of the power curve, looking down at all other languages. And Lisp practitioners are sure they're looking down, and they'll tell you why. "How can you get anything done in Haskell? It doesn't even have macros."

But Haskell programmers are sure that they're at the top of the power curve, and that when they look at other languages (including Lisp!) that they're looking down. And they know why they're looking down. "How can you get anything done in Lisp? It doesn't even have a decent type system!"

I have in fact heard almost the exact argument of PG's essay before, from proponents of a heavyweight CASE tool.

This fact - that proponents of two languages are both sure that they're looking down when they look at the other language - tells us that something is very wrong with PG's idea. The problem is the idea that all languages can be ranked on a one-dimensional axis labeled "power".

To see what's wrong with this idea, look at hardware. We know what "power" means there - it's MIPS. Until you think a bit more. Then you realize "Well, we've got that floating-point code, so we have think about FLOPS, too. And there's that one data set that won't fit into cache, so we need to worry about memory bandwidth... unless we can trade off some clock speed for cache size..." Suddenly "power" has at least four dimensions: MIPS, FLOPS, cache size, and memory bandwidth.

Then some joker comes along and says, "What I mean by "power" is the ability to run for eight hours without having to charge the battery"; that is, "power" is something like the reciprocal of watts consumed.

Back to software. It isn't just power of the language, it's power to do something - to write the program you're trying to write. What language is going to make that easiest, all things considered - the language itself, libraries, ecosystem, familiarity of team-members, ease of learning? Pick that language, because that language is "the most powerful" - for your situation.

[BBL]

I've used ruby to make a responsive web app and feedback on the product signifies that users like and trust the seemingly clunky design elements.

I believe the founder of Facebook previously said that the company's early over-reliance on HTML5 was his biggest regret. The tradeoff paid off in terms of speed and agility. The use of the color blue was also quite smart.

Note: My hidden agenda with this post is to increase my karma points.