Always fun to point out is that the NY federal reserve switched their full scale model of the economy to Julia last year[1]. It's also open source, so data scientists interested in macroeconomics can feel free to play around with it.
Moreover, quant-econ, an academic entity dedicated to scientific programming in the social sciences has a few notebooks in python and Julia [2]
Models are just datastructures and algorithms; presumably, you mean there is an implementation of those, and that the Fed uses that implementation as the 'official' one.
That's fine, but I prefer my models to be language-independent.
Most financial firms need their quants to worry less about the new programming language hotness, and more about moving entire systems off unbelievably complicated Excel spreadsheets.
I really love Julia and see a bit potential outside of the scientific computing. I used to do all sorts of automations, smaller throw away programs etc in Python and Ruby, but I find Julia much more effective to work with.
It also feels like a smaller language in many ways. You don't have to keep as much in your head, which makes it more effective to keep up with.
The killer features for me, automating a lot of tasks is the awesome integration with the shell environment. Running and combining Unix commands is very elegant and getting hold of the output is done really smoothly.
Secondly what I think saves a lot of time is that it really has ALL BATTERIES included. With Python that feels more theoretical. With Julia you just start writing the name of a function you think should exist in the REPL and it pops up. With Python you got to start guessing which package it might be located in. And that goes for stuff you use all the time like string manipulations, regular expressions, file reading and writing, process input and output etc.
With Python functions often end up having unnatural names because you can't have naming conflicts. Julia handles functions with the same name but different argument types elegantly.
Also there is no schizophrenia wondering if something is a function or method on an object. Everything is a function so that is easy.
I would also like to use Julia more for "throw away" scripts and general automation. Unfortunately, startup time (and sometimes compilation) make this a less fun endeavor than with Python etc.
Julia is a very interesting language but big data processing is not a matter of the programming language itself, but how your platform and your architecture is put together.
And nearly always your bottleneck is not the language itself..
Depends on whether you're optimizing for latency or throughput - and of course, in some systems, that decision changes on an instant-to-instant basis. For throughput, indexing and shuffling is probably going to be the bottleneck, and that's dependent on network and filesystem I/O. But for the low-latency path, choice of language absolutely matters, so if you want to be able to share code between the two you end up needing both a fast language and fast architecture.
I think that if this were true, the data science libraries in Python wouldn't all be written in C under the hood, linking to Fortran, etc. Language can have a huge impact on performance with big data.
Julia is just a small step forward. It still has performance problems where the JIT can't help much - like the DataFrame structure, which is effectively a black box and therefor hard to optimize.
I agree that this is generally true, but Finance is sort of a special case. In a world where firms pay the premium for colocated servers and direct data feeds in order to shave off a few milliseconds of latency there is also a strong push to use the vary fastest language possible + have the very best possible architecture, all to shave off as many extra milliseconds as possible.
I'm really interested in learning more about data science. Currently attending moocs to learn R but I don't have much free time and it's a slow progress. Currently I work as analyst for an environmental consulting firm but I always wanted to work in something related to quants. I know I'm probably being naive but I want to change careers. Do you guys recommend that I keep learning R or should I take a closer look to Julia? Or both? I don't have a programming background but I do know statistics and Excel. Edit: fast, concise and useful comments in a matter of minutes, I love HN. Thanks!
Become fully confident & proficient in a single programming language before hopping around. If you get to the point where you can build a full application from scratch and answer other peoples' questions on SO or IRC with one, you'll be able to easily get up to speed with another should the need arise.
There are many more resources for learning R, given that the language is older. These resources have become much better in recent years and I think there has been no better time to learn R. Probably the best introduction to data science in R is Hadley Wickham's latest book: http://r4ds.had.co.nz
Learning your second language is much easier than learning your first. You'll be more familiar with programming concepts and terms that make it easier to search for how to do X in that language.
Julia is still relatively new and not quite ready (in my opinion) for new programmers. Things change relatively quickly and I think that could add a lot of confusion that gets in the way of learning.
That being said, I think Julia is a pretty great language. I really love the way it has grown and I'm excited to see how it matures.
R is great for producing scientific papers, charts, and doing a vast array of "standard" statistical analyses. I have found python to be much nicer because it's more of a "real" programming language, but the numpy/scipy/pandas stack make it very easy to use for data science and financial analysis also.
>I always wanted to work in something related to quants
Then learn Python, it is mature and has way more study material.
Not to mention Machine Learning libraries like TensorFlow, Scikit Learn & math libraries like Numpy, Pandas, Matplotlib?
Everything there is just set.
I once tried to learn Julia, but the community is just to small to get help at times. It is a work in progress but I recommend you to keep an eye on It.
Conclusion:
Think of Julia as a future successor to Python, but not a current replacement.
My beef with Julia when I last tried it out was the lack of flexibility within the 'data munging' sphere. The assumption appeared to be that all of your data would come in to as UTF-8, whereas in my org we need strong support for Latin1/cp1252, and UTF-16LE BOM. Is this something that has changed recently, or are these traders are economists just working with more sane data?
AbstractStrings (the generic string abstraction) can have any encoding – only the standard built-in String type is UTF-8. If you've got data in another encoding, use the StringEncodings [1] package to read it.
Not a language issue per say then. It shouldn't a problem to add support for this. Julia is even designed from the start to work with Strings of different types. Something not common in most other languages I've worked with.
Most string functions operate on AbstractString types. You can create whatever subtype you like implementing alternative string encodings.
I am not a quant programmer, but last summer I did a ten day job for an old customer in Julia and I had to learn the language as I worked. I thought the expressiveness and conciseness of the language was good and I liked the way the package manager used github projects; easy learning curve and REPL based development.
As a quant, I can't say I've heard of a single quant I know who uses Julia. It very well could be promising, but firms will be very hesitant to switch from Matlab, R, Python, C++, and VBA
When Julia has an environment as good as Rstudio and a dashboard as good as Shiny, and a Spark interface that is as good as Sparkler and R-Spark then I'll be up for it.
Writing good code in R is an effort, it's possible but the language doesn't make it easy. Julia makes me write good code off the bat and facilitates writing excellent, elegant code.
This is how I tend to feel about all non-R things; I've been using R for long enough to dislike a lot about the language, but the environment and tooling is too good to give up.
Letting R be the first language I've gotten pretty good at is really spoiling me.
I really liked Julia (and Juno) when tried it last year, but it was too math-aware as all [actually good] math-originated programming environments. Hope it will break that circle and get some general-purpose attention in nearest future. For me, it is very promising even without that HPC or whatever.
Yes, there's been some scientific workloads that used Julia/MPI to great effect. I don't think the paper is out of review yet, so probably can't link to details, but expect news on that front very soon.
I doubt that. It's certainly the trend and a lot of new code is produced in Python. But most models will run on R/Matlab/C++/C#. I'd guess that most quant departments have less than 10% of their code in Python.
But Python is the modern way to go, so every shop stresses that they use it and they expect every new hire to master it.
So how come not golang instead of Julia? I don't have anything against Julia myself but it seems like go with its maturity, easy cross compilation, static compilation and easy IPC/multithreading it would fit the bill quite well. Granted the GC is a bit slow but should be improved with the next release.
Given Julia's pre-1.0 status, general immaturity and good Python interop wouldn't a saner approach be to use Python as your primary language and Julia for whatever needs to be fast?
it's almost the perfect language. The breaking issue for me was the choice to count from 1 (yes, I know there are pro arguments for counting from 1, but every other language I use doesn't do that).
I've never used Julia but it reminds me of Go in that it attempts to replace Frankenstein systems that mix low level languages and high level languages/scripts, with a single modern, high level language with sufficient low level features and performance.
However, in order to do so, they have to give up the generality of either language, and focus on a particular domain (numerical computation and web servers respectively). And because of this, the language never develops a big enough ecosystem. Because of this, I think Python combined with native C libraries is ultimately the better way forward and will be more successful.
[+] [-] VodkaHaze|9 years ago|reply
Moreover, quant-econ, an academic entity dedicated to scientific programming in the social sciences has a few notebooks in python and Julia [2]
[1]https://github.com/FRBNY-DSGE/DSGE.jl
[2] http://quantecon.org/notebooks.html
[+] [-] amelius|9 years ago|reply
Is there some way to play with these models? Or do you need lots of input data, which isn't freely available?
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] dekhn|9 years ago|reply
That's fine, but I prefer my models to be language-independent.
[+] [-] s_q_b|9 years ago|reply
[+] [-] jernfrost|9 years ago|reply
It also feels like a smaller language in many ways. You don't have to keep as much in your head, which makes it more effective to keep up with.
The killer features for me, automating a lot of tasks is the awesome integration with the shell environment. Running and combining Unix commands is very elegant and getting hold of the output is done really smoothly.
Secondly what I think saves a lot of time is that it really has ALL BATTERIES included. With Python that feels more theoretical. With Julia you just start writing the name of a function you think should exist in the REPL and it pops up. With Python you got to start guessing which package it might be located in. And that goes for stuff you use all the time like string manipulations, regular expressions, file reading and writing, process input and output etc.
With Python functions often end up having unnatural names because you can't have naming conflicts. Julia handles functions with the same name but different argument types elegantly.
Also there is no schizophrenia wondering if something is a function or method on an object. Everything is a function so that is easy.
[+] [-] leethargo|9 years ago|reply
[+] [-] sebg|9 years ago|reply
* http://juliacon.org/2014/
* http://juliacon.org/2015/
* Julia Con 2015 India - https://www.youtube.com/playlist?list=PL279M8GbNseuhXmZk9rWM...
[+] [-] optimali|9 years ago|reply
Overview - http://julialang.org/blog/2016/09/juliacon2016
[+] [-] agounaris|9 years ago|reply
And nearly always your bottleneck is not the language itself..
[+] [-] GeneralMayhem|9 years ago|reply
[+] [-] staticassertion|9 years ago|reply
Julia is just a small step forward. It still has performance problems where the JIT can't help much - like the DataFrame structure, which is effectively a black box and therefor hard to optimize.
[+] [-] importantbrian|9 years ago|reply
[+] [-] pitaj|9 years ago|reply
[+] [-] Ftuuky|9 years ago|reply
[+] [-] mi100hael|9 years ago|reply
[+] [-] ellisv|9 years ago|reply
Learning your second language is much easier than learning your first. You'll be more familiar with programming concepts and terms that make it easier to search for how to do X in that language.
Julia is still relatively new and not quite ready (in my opinion) for new programmers. Things change relatively quickly and I think that could add a lot of confusion that gets in the way of learning.
That being said, I think Julia is a pretty great language. I really love the way it has grown and I'm excited to see how it matures.
[+] [-] strictnein|9 years ago|reply
Data Smart: Using Data Science to Transform Information into Insight
https://www.amazon.com/Data-Smart-Science-Transform-Informat...
All the work is done in Excel. I enjoyed the book quite a bit. Author is Chief Data Scientist for MailChimp.
[+] [-] cwmoo740|9 years ago|reply
There's a (very) quick intro here:
http://lectures.quantecon.org/py/pandas.html
You can also look into quantopian for practice, but I don't recommend actually trading money.
https://www.quantopian.com/
[+] [-] SixSigma|9 years ago|reply
https://www.coursera.org/learn/julia-programming
[+] [-] max_|9 years ago|reply
Then learn Python, it is mature and has way more study material.
Not to mention Machine Learning libraries like TensorFlow, Scikit Learn & math libraries like Numpy, Pandas, Matplotlib?
Everything there is just set.
I once tried to learn Julia, but the community is just to small to get help at times. It is a work in progress but I recommend you to keep an eye on It.
Conclusion: Think of Julia as a future successor to Python, but not a current replacement.
[+] [-] bo1024|9 years ago|reply
[+] [-] blitzd|9 years ago|reply
[+] [-] StefanKarpinski|9 years ago|reply
[1] https://github.com/nalimilan/StringEncodings.jl
[+] [-] z92|9 years ago|reply
[+] [-] jernfrost|9 years ago|reply
Most string functions operate on AbstractString types. You can create whatever subtype you like implementing alternative string encodings.
[+] [-] mark_l_watson|9 years ago|reply
[+] [-] hodder|9 years ago|reply
[+] [-] sgt101|9 years ago|reply
Writing good code in R is an effort, it's possible but the language doesn't make it easy. Julia makes me write good code off the bat and facilitates writing excellent, elegant code.
[+] [-] claytonjy|9 years ago|reply
Letting R be the first language I've gotten pretty good at is really spoiling me.
[+] [-] sdegutis|9 years ago|reply
[+] [-] wruza|9 years ago|reply
[+] [-] arcanus|9 years ago|reply
I've heard plenty of buzz about it in the HPC community but haven't had a moment to play with it yet.
[+] [-] KenoFischer|9 years ago|reply
[+] [-] Xcelerate|9 years ago|reply
[+] [-] fredcy|9 years ago|reply
That says, it's less important than you could possibly state.
[+] [-] williamstein|9 years ago|reply
[+] [-] dx034|9 years ago|reply
But Python is the modern way to go, so every shop stresses that they use it and they expect every new hire to master it.
[+] [-] tekknik|9 years ago|reply
[+] [-] whyrt12|9 years ago|reply
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] cutler|9 years ago|reply
[+] [-] jefferson666|9 years ago|reply
could anyone give a performance test of it? many thanks
[+] [-] dekhn|9 years ago|reply
[+] [-] rrock|9 years ago|reply
http://docs.julialang.org/en/latest/devdocs/offset-arrays/
[+] [-] farright|9 years ago|reply
However, in order to do so, they have to give up the generality of either language, and focus on a particular domain (numerical computation and web servers respectively). And because of this, the language never develops a big enough ecosystem. Because of this, I think Python combined with native C libraries is ultimately the better way forward and will be more successful.