top | item 10197197

Bloomberg Runs on 25M lines of Fortran (2006)

160 points| charlieirish | 10 years ago |etrading.wordpress.com

149 comments

order
[+] apaprocki|10 years ago|reply
Sigh. Bloomberg-er here -- this article is old. There is certainly a lot of Fortran remaining in sections (you don't rewrite working code with C linkage in another language just because -- you rewrite it if you have something to gain), but it is nowhere near as bad as this old article makes it sound. Many teams are C++ only and have been for a long time (see our open-source code on GH for a sample). Keep in mind that if you deal in absolute numbers you are missing the bigger picture -- the actual real number will be much smaller and it will be an even smaller percentage of the total codebase.

edit: Can mods put (2006) in the title? Seems reasonable...

[+] raldi|10 years ago|reply
Seconded. I worked at Bloomberg from 2003-2007, and over the course of that time I think I needed to change about five lines of Fortran total. It was mostly used in "if it ain't broke" systems that weren't seeing a lot of change; everything under active development had long since been rewritten in C, C++, or a higher-level language.
[+] spacecowboy_lon|10 years ago|reply
What REAL advantage does the switch to C++ from Fortran have for you?

And NO not knowing fortran doesn't count if at 18 at my first job was told oh hear is a book on fortran learn it your graduates with a fancy CS degree can do the same.

[+] zerr|10 years ago|reply
Regarding C++, what are percentages of "90s style C++" vs Modern C++11/14?
[+] forgetsusername|10 years ago|reply
>Sigh. Bloomberg-er here -- this article is old.

Why be offended, rather than excited to get to come in here and to expound on what you do for a living? It's what makes the comment section great.

[+] radmuzom|10 years ago|reply
The article seems to assume that there is something "wrong" with running on 25M lines oF Fortran. It does not provide any technical reason why Fortran is not the right tool for the job. Why does this need to be "Web 2.0"?
[+] Muted|10 years ago|reply
I'm also confused about this. I don't understand what is wrong with Fortran. In school we learned Fortran 2003 for numeric simulations etc. For my master thesis I programmed in C and there were a lot of things in C that I missed from Fortran. 1) Fortran is easy to learn, no need to know pointers etc 2) very math friendly, easy to extract subarrays/submatrices etc, no need to pass around array lengths etc. matrix's are really easy to deal with. I remember in C we ended up rewriting all two dimensional arrays (matrices) to column major arrays cause it was a pain to deal with (or maybe we were noobs) 3) useful built in functions such as matrix multiplications, in C you end up writing a lot of functions to do what seem like very simple things 4) Great math libraries such as BLAS/LAPACK 5) Seems to me like it less error prone than C (spend a lot less time debugging)

This is just my experience with it. Somewhat to my surprise I really ended up liking Fortran, and the new code doesn't look at all like the old code. For what we did it was also slightly faster than C. If I were to write a numeric program I wouldn't think twice about what language to use. I feel like Fortran is an undervalued language for calculations. But then again, I am by no means an expert.

[+] DrJokepu|10 years ago|reply
I client of mine still has a fairly large piece Delphi of code in production and I must say it's a bit of a liability for them. An example of a technical problem they face is getting the particular version of Delphi they use work with TLS (and that's becoming a problem now as everyone's starting to disable SSLv2 & SSLv3). The fact is that they simply wouldn't face this kind of problems if they have ported their code to something like Python, C# or Java.

On the non-technical side of things, they also struggle with hiring developers who are capable of (or willing to) work with Delphi, especially junior developers. A significant portion of their staff also doesn't know how to work with Delphi, so reassigning resources to match the needs of the business is often a challenge.

I imagine these issues would probably apply to Fortran as well.

[+] amyjess|10 years ago|reply
I think a lot of people assume Fortran == FORTRAN 77, which is a terrible language without even full support for structured programming and with an annoying fixed-field syntax. There are good reasons to hate on it.

Most Fortran hate is actually F77 hate.

Fortran 90, on the other hand, is basically C with a different syntax. F90 (and newer: F95, F03, F08) really isn't a worse language than C, and there's no reason to hate on it when you wouldn't hate on C, but a lot of people don't know it exists because they equate Fortran with F77.

[+] tempodox|10 years ago|reply
Never underestimate the power of Fashion. And if your server software doesn't run on Node.js, you're not hip enough anyway.
[+] Marazan|10 years ago|reply
The problem is that the code base is byzantine with developers hired who don't have good Fortran knowledge.

There are a million different versions of the same function each with a minor tweak to get the "right" behaviors labeled func_name_1 down to func_name_1000000 because everyone is too scared to fix the original function as they don't know what depends on it.

It's not that Fortran is the right tool for the job it's that it has terminal momentum.

[+] dagw|10 years ago|reply
It also depends very much on which Fortran we're talking about. Fortran has evolved more than just about any other language out there, and Fortran77 (not a very good language) is very different from Fortran08 (a pretty great language).
[+] dgritsko|10 years ago|reply
Ironically, it was the use of the term "Web 2.0" in the article that signaled its age to me.
[+] Shivetya|10 years ago|reply
Most people will point out the support issue, namely getting knowledgeable people to work on it might be difficult. In the case of the code I have worked on while we may rewrite a working system in a newer language it is more typical to wrap known bullet proof functions in layers to present it in new formats; web/mobile/etc.
[+] hawleyal|10 years ago|reply
Technical debt. Do you still have a VCR? Isn't it a bad idea to assume VHS is a good medium to archive?
[+] ugh123|10 years ago|reply
Two reasons are maintenance and hiring. How many talented Fortran developers are out there now (and living in NYC) available to fix and maintain code that your entire business depends on?
[+] Shorel|10 years ago|reply
I agree completely.

My own take on that was: "Wow, they surely must do a lot of performance sensitive stuff to use the fastest language available for number manipulation. Kudos to them."

[+] ig1|10 years ago|reply
(In a previous life I worked at Bloomberg)

I don't remember the exact figures but 25m lines of Fortran seems plausible, but it was still the minority of the code base. The majority at the time was in C and C++, with probably a few million lines of Perl and Javascript as well.

Plenty of the Fortran dates from the 80s, rewriting it in a modern language would be a huge project which lots of risks and limited upside. No significant new fortran has been written for a long long time (>10 years).

It's also a mistake to think of Bloomberg as a single piece of software, it's closer to an app store, there's are thousands of specialist apps (functions in Bloomberg terminology) and most users only use half-a-dozen, but which half-a-dozen varies significantly between users.

This makes it very hard to displace wholesale. There are individual pieces a competitor can go after but identifying a subset of functionality that's enough to convince users to switch is hard.

There's also a lot of stickiness which goes beyond pure functionality. Network effect and brand are also key parts (having a Bloomberg Terminal is a status symbol).

[+] PaulHoule|10 years ago|reply
I have been looking at some projects at Bloomberg from the outside and even those the projects are interesting I don't think I'd want to work their because of bureaucracy and inertia -- it's the kind of place where you'd need to get approval to install a text editor on your machine.

I have found that often "high profits" get in the way of customer service because they are a disincentive to "quality is free" thinking and make it possible to sustain the unsustainable for way too long.

To take an example, Cable companies are so profitable that they think nothing of the cost of replacing cable boxes that break, or of excessive truck rolls. These not only cost money but they anger consumers. My mother-in-law quit cable in disgust and switched to OTA TV after she had three cable boxes burn out in three months and would have to go stand in an long line to return it and pick up a new one.

Quality is free and screwing up is expensive -- even if you can afford to screw up it costs you customers and it costs you employees. But again, turnover is no problem if you are making enough money you can afford to spend 5x what things should really cost.

[+] melling|10 years ago|reply
This article is from 2006. I imagine something has changed in 9 years.
[+] retube|10 years ago|reply
You clearly haven't worked in a large enterprise!
[+] raldi|10 years ago|reply
The information in this article was wrong even back in 2003 when I started there.
[+] sean_the_geek|10 years ago|reply
thanks for pointing that out! Completely missed it.
[+] drglitch|10 years ago|reply
BBG's competitive advantage is not data - in fact, it's pretty mediocre. Any reasonably large player sources their own data feeds due to too many holes in the former.

BBG's killer features are support and _chat_ - chat with other _trusted_ counterparties who also paid the admission price.

[+] beezle|10 years ago|reply
It would have been interesting to know if they are using Fortran 95 or the more recent (and very modern) 2003. People who like to beat on Fortran rarely know that it has evolved quite a bit since '66 and '77
[+] matthewaveryusa|10 years ago|reply
I worked at bberg for 4 years. Changed about 2 lines of fortran code. Some people do more, others so less, but the norm is you'll spend a sliver of your time in fortran.

Great place to work, too bad it's in NYC

[+] baldfat|10 years ago|reply
A little Wikipedia information:

Market share has grown since this was written in 2006.

Market share in 2007 26% and 2011 30%

Money: 315,000 Bloomberg Terminal subscribers (2 year subscriptions) worldwide at $20,000 per user = 6.3 BILLION dollars a year!

EDITED for spelling

[+] edu|10 years ago|reply
The article is from 2006, should be noted on the title. And, anybody knows if the situation is still the same?
[+] stevoo|10 years ago|reply
I have to disagree with having a company actually coming in and taking out everything that bloomberg has been building the past years. This is an extremely hard market to get it. Bloomberg is extremely dedicated at what they do and they do not take competition lightly. They do use there massive power to make the life of competition as hard as they can.

As for Bloomberg there terminal and all of the services that they provide is top notch. I have a lot of friction with all of there data, and this is by far the best i have worked with.

[+] smanzer|10 years ago|reply
Unhappy academic here - codebases with millions of lines of Fortran are very prevalent here as well. It is both better and worse than a lot of people think; like a lot of the other comments said, that code is rarely touched, but when you need to touching it can be very painful. Modifying old Fortran is rough, mostly because of that awful, awful IMPLICIT keyword. Though I will say that the native multidimensional array support does make certain sections much easier to read than corresponding vanilla C code.
[+] pma|10 years ago|reply
Bloomberg terminal and Thomson Reuters are certainly ripe for strong competitors.They are basically operated as monopoly for decades.Yc,developers and others should take on this challenge. http://www.nytimes.com/2015/09/10/business/dealbook/the-bloo...
[+] KaiserPro|10 years ago|reply
Traders value consistency and dependability.

Its not like you can throw together some trading system with an eventually consistent datastore in the back.

the entire finance industry is based on trust. Part of that is trust that bloomberg is correct.

Loosing a couple of trades without an audit trail crashes markets.

[+] throwaway5752|10 years ago|reply
First it's Thomson Reuters. Second, YC has little chance in this market which requires large capital for the data feeds, and has enormous incumbent advantages. Third, FactSet is a player there, it's not even a duopoly.
[+] brobinson|10 years ago|reply
I recently read about this company in the NYTimes which is trying to do just that: https://www.money.net/

$99/mo/user vs $21,000/year/terminal. Wow! I wonder what other kinds of entrenched, expensive products like the Bloomberg Terminal are in need of competition. I think this requires industry-specific experience to know, however.

[+] Keyframe|10 years ago|reply
That business is more about connections than anything. While tech stack is great, it's all about data feeds. Fast, exclusive, reliable data feeds.
[+] coliveira|10 years ago|reply
YC style companies are great for consumer software, but BB is directed at financial companies, which are very conservative and put a lot of value in support. There is no way to win in that space by being cheap, you need a lot of investment to provide mature code and high-quality support by real people.
[+] aaronkrolik|10 years ago|reply
I'd say that Slack is actually bloomberg's biggest competitor at the moment.
[+] ZanyProgrammer|10 years ago|reply
I'd argue the Westlaw portion of Thomson Reuters is certainly ripe for competition.
[+] monopolemagnet|10 years ago|reply
By comparison, this is 33% more LoC than a BWR/PWR nuclear reactor (Monte Carlo method / quadruple integral (⨌) equivalent) simulator product formerly known as CORETRAN-01.
[+] monopolemagnet|10 years ago|reply
Finite element analysis modeling is really hard, even for nuclear engineers whom can code.

A lot of scientific control system apps get built with LabVIEW or Agilent VEE. If it makes beyond the niche consultancy stage, such things are often (re)written in whatever an engineer happens to know: C, Python, Haskell, LISP, [favorite language religion here].

Edit: BTW, just came across a supposed successor to VMEbus [0] (a standard industrial data bus): VXIbus [1]

0: https://en.wikipedia.org/wiki/VMEbus#/media/File:VMEbus.jpg

1: http://www.vxi.org/specifications.html

[+] zimbatm|10 years ago|reply
When the compiler code size is a fraction of the whole it would actually make sense to improve on it instead of rewriting everything. Like Facebook did for PHP and their HHVM. Obviously Fortran doesn't have a speed problem but it could be improved in other ways like static analysis and nicer syntax notations.
[+] protomyth|10 years ago|reply
Converting to later versions of the Fortran standard would get you a long ways there. Plus with all the focus in 2003 & 2008 with C interoperability, you can plug in later open source libraries.

[edit: I wonder what a modern version of RATFOR would look like]

[+] shitgoose|10 years ago|reply
In Fortran one can add two matrices: A=B+C. Try it in C++ (NumPy is better that way, but still this is an extension to a rocky foundation). The amount of freely available high quality math/statistical libraries for Fortran is unmatched. Why in the world would you give up on this just because you can use Google account? Google Payments for settlement?? WTF? Settlement is a bit more then credit card charge or moving money between checking accounts. Google Fiance as a ticker plant?? Yea, right. Even Yahoo provides options data, Google has just basic stock prices. Compare it with BBG that has everything. Build terminal in Web Kit? Why???

Every time I see suggestions like this, I remember minions, rushing from one framework to another - "koonga la mala makuna, koonga!".

[+] jamesrom|10 years ago|reply
And there are 6 million rivets in the Sydney Harbour Bridge.
[+] josephmx|10 years ago|reply
This reads a bit like the old "Google will rule the world!" blog posts, doesn't it?
[+] tempodox|10 years ago|reply
I don't know how comparable this is, but JaneStreet seem to be quite happy with OCaml. I for one would give that some consideration before actually going C++. And that's ignoring the question why they would depart from Fortran in the first place.