The main issue I found in algo and financial aspects of programming is that the market is a zero sum game, and my intro knowledge of finance and algorithms, even when I know python, are no match for MIT PHD Quants who does it full time. There's no real way to compete with that, and therefore I would lose money, even if the data showed it might be successful in the future, firms and full time workers on algo trading would simply be faster, more focused, have more funding, and be able to quickly and constantly adapt at the scale an individual could not.
So despite the fact that the subject is interesting, I'd consider it a waste of time to try and gain anything but a basic understanding of the industry and how algo trading works.
An economist and a normal person are walking down the street together. The normal person says “Hey, look, there’s a $20 bill on the sidewalk!” The economist replies by saying “That’s impossible- if it were really a $20 bill, it would have been picked up by now.”
I agree with most of what you're saying in principle, but it is very possible to identify alpha or acquire an edge that institutional market participants don't have. If you have access to data that most of the market does not, you can effectively trade on it. You can also effectively trade on a novel insight on a combination of data sources.
There is a lot of information asymmetry, and an individual is capable of capturing that without requiring a PhD or the resources of a large firm. That's not to say it's easy per se, but it's not hopeless. It requires special expertise or an unconventional approach.
Otherwise I agree that most people probably shouldn't attempt it (for risk tolerance reasons).
You are oversimplifying and overcomplicating at the same time. If it was an easy/obvious zero-sum game, we'd not have people going into finance at all.
MIT does not teach "HFT", those "MIT PHD Quants" are just as untrained in finance as you are. The rest is math, something that requires time to study but not necessarily a certificate. And creativity. I've only seen a single HFT-algo that made me say "wow, that's creative"; all others were just straight implementations of finance "wisdom".
Why do you say it's a zero sum game? If I've learnt anything from my time in finance, it's that the market is definitely not zero sum. The prices you see represent sentiment, not a hard valuation, and someone isn't necessarily losing when you gain. If you really think about it, even entire economies aren't really zero sum as our method of valuation is intrinsically subjective!
You are painting with an extremely broad brush. Before you give up on this idea, just try this exercise: enumerate the markets and financial instruments that you could potentially trade in, and characterize the current trading environment in that market.
An ultra-HFT liquidity provider being profitable has little bearing on the potential profitability of quantitative trading on significantly larger timescales. There isn't sufficient volatility in most products for these actors to eat the lunch of actors who have alpha on trades that can hold for significant periods.
Even if it's a zero sum game (which it's not), most participants are not quant/algo based. As long as you can beat 50% percentile, you can make a profit.
Sure you may not be as profitable as top quant companies, but do you really mind that much?
Sorry, but this is like saying those CalTech PHD in Computers Science are the best, so no point for me to go into that field...
This is like everything in life, not easy, but if you work on it it'll pay out in the end
If the market is a zero sum game, then for every winning trade, there must be a losing trade. Obviously some teams must be winning consistently, which implies that some are losing consistently. This says that the consistent losers go out of business. So who are the winners going to trade with? By contradiction, it is not zero sum.
I'm curious to know who the loser was with the cryptocurrency rally this year?
You've forgotten you can introduce new units of whatever is being traded and quite often is & there are additional complications such as dividends, stock options, etc which makes your oversimplification lacking substance.
Personally, I struggle to see the competitive advantage Quantopian brings. They use retail brokerage platforms to facilitate trading, which rules out anything close to HFT. Then, they are tied to any financial data vendor (Morningstar in this case) to not offer too much visibility on the underlying data. As others have mentioned, this makes it tough to validate aspects like adjusted vs. as reported earnings, how delistings are handled, etc. From my experience, getting/making sure the data is accurate is a ton of work, even if it is from good sources and you can see all the actual data. The moment an investor/trader on the platform gets traction is the moment they want Compustat data, exchange data, Bloomberg for fixed income, and will trade through Instinet/Flextrade, etc. The moment the platform is successful is the moment Morningstar could pull the rug out from them. If someone has better understanding or knowledge on Quantopian in particular, I'd be interested to hear why.
My understanding is that Quantopian has no interest in being a platform for HFT. It's there to democratize trading strategies and test them in a sound way. They do a lot to have a good backtesting platform and clean the data that is available in the platform. If I wanted to automate a strategy, I'd have to figure out the Robinhood API and basically recreate what they have. Instead, I can use their platform, their backtests, their free data, and build strategies that efficiently lose money.
I think the other goal being that if you create a profitable strategy, you can enter it into competitions, trade with other people's money, and make a profit. I think the platform is a value add, I have an account and use the research notebooks. Some day when I more spare time I'll make some trading algorithms.
They're searching for "alpha" strategies. Then they buy/share profit with the creator... I have a hunch that they are feeding these good strategies to some type of ML in order to become more efficient and profitable.
So, I got semi-seriously interested in this around the end of last year.
My takeaway is: it's not about implementing a couple of trading strategies. It's about implementing a pipeline that rapidly allows you to test what-if scenarios.
I might have like 10 ideas a day for strategies. How many of those can I rigorously validate per week? What about variations? I.e. tuning various hyper-parameters? Combinations?
How quickly can I recombine data from various sources into exactly the layout I need to train and test this or that strat?
Me:
1) Build some simple forecasting NNs.
2) Realize I need to be able to generate and test ideas waaay faster. Start working on infrastructure.
3) Get impatient -- get some "bright" ideas, do some manual trading.
4) Make some money on the first day, lose double that on the next two days.
5) Decide to ban myself from manual trading as obviously I'm an addict. Resolve to only do algorithmic trading.
6) Back to coding infrastructure. Get bored ... do some manual trades.
7) After losing around $20k, stop the trading madness, and go back to buying and holding great tech companies.
QuantConnect recently announced full python library support; and we have launched https://www.quantconnect.com/tutorials to help people write quantiative strategies in Python.
QuantConnect & LEAN gives you ability to do tick->daily resolutions; for equity, morning-star, future, option, forex and cfd trading - all with a fully open source project which includes samples of data to get you started.
The grunt work is still done in C# so its faster than other full python based backtesting engines. Edit: I'm the Founder @ QC.
1. Any chance of a Robinhood integration à la Quantopian?
2. How are the architectural revisions[0] coming along?
Also some totally unsolicited feedback:
If I'm being completely honest, I found it difficult to get going with QC. The documentation is decent, but there's not enough to avoid having to review LEAN source right off the bat. The examples also tend to mix helper classes with lower-level functionality, and that can create confusion.
The framework itself feels a bit over-reliant on OOP. Some aspects feel too tightly coupled, others too little. Obviously LEAN has been around for many years now, so architectural baggage is perfectly understandable.
A total rewrite I'm sure isn't feasible, though I'd suggest the following ethos in any case:
a) Design primitive user-accessible data structures with virtually no inbuilt functionality.
b) Build low-level components that operate using those data structures.
c) Build high-level components that compose low-level components.
d) Allow users to author their own components, and to compose components of any type however they see fit.
Pretty sure you're already on that track in a sense, so it's good to see things headed in the right direction. What keeps me from writing a custom framework is the data, the fact QC does a ton of grunt work, and that it's well-tested.
tl;dr Please break apart the monolithic QCAlgorithm class as much as you can! :)
When promoting your company, please say so, also on QuantConnect one cannot actually see the data, so there is no way to verify how good the data is...
On a side note, the market continues to do well and I've been noticing this trend of active-trading, real-estate investing gurus crawl out of the wood work selling services.
Please do not try to trade actively unless that's your full-time job. Passive investing using index funds is definitely not sexy, but it gets the job done.
> Passive investing using index funds is definitely not sexy, but it gets the job done.
Only if you have a significant amount of capital to play with. People looking into active trading are doing it because they don't have the necessary capital to make passive investing meaningful; they're looking for much much larger returns that you can get from an index fund, which necessarily comes with more risk. Traders are gamblers.
I went to the Quantopian conference for their basic training on algorithmic trading. This blog post was pretty much what they covered, intro to pandas and a simple strategy. There is a lot of educational material on their site too (which is what you ended up getting in the paid training).
My biggest thing with the Python for Finance books - I know Python, I want to learn finance. All these books are the inverse of that, for people who know finance and want to learn Python. There is a good site for quantitative economics [1] that has tutorials in Python and Julia. I would love a mathematics of finance book that had the examples in Python.
those are indeed good lectures but they are not math for finance per se - more like dynamic programming for econ/finance. my impression is that finance as such is a hodge-podge of econ/accounting/business.
The problem with learning finance that it takes a lot og effort. A good place to start would be taking the 3 CFA exams. After about 1000 hours of study, you will have a basic grasp of finance.
Did you find the conference worth it ? I was thinking of going to it as well or even going to the online version since it's a lot cheaper but ended up not taking part.
This is nothing more than gambling. Let's say you have 20k to play with. You would be far better off in the long run to put 15k in Wealthfront and use the other 5k as a bankroll of 25 buyins for 1/2 no limit holdem to learn the game.
The article mentions a few of the pitfalls of backtesting, but it does not mention one of the best tools at your disposal in backtesting: Walk Forward Optimization/Analysis
I won't say I have "experience" but quantconnect.com is as good a platform as I've found for C#. Their engine is open source so you don't have to rely on them either, though you'll need to provide your own data if you run it yourself.
I have a mean reversion strategy based on the comparing the results of several types of sentiment analysis in real time. It earns between 3 - 15% in options trading every few days with a win rate of around 70% and an average holding time of a few hours. Stop losses are at -70% or so.
Algorithmic trading is very difficult, but it is empirical. Information asymmetry exists in the market, and if you can capture it you profit. I don't know how capacity constrained my strategy is (I think it would be difficult to work this strategy profitably with over $10M depending on how much effort you put into execution; my execution is unsophisticated and I'm working with two orders of magnitude less than that), but it's working.
Algo trading typically utilises technical analysis which is basically patterns proven to repeat in markets for a variety of fundamental reasons, or fundamental analysis (e.g. algorithmically valuing and pricing options based on underlying fundamental data, sentiment analysis, etc.), and buying/shorting as appropriate. It is based on scientific methods: empirical evidence being used to validate hypothesis that produce results that can be shared and repeated by others.
Astrology is story-telling with no scientific grounding.
When filing for its IPO in March 2014, it was disclosed that during five years Virtu Financial made profit 1,277 out of 1,278 days, losing money just one day.
It is a great article, but why on earth someone will use a service like Quantopian or similar service?
They are your competitor and who will prevent a disgruntle employee or a hacker to steel your successful trading strategy?
Just buy some data from eBay, you can get 20 years of historical stock market data for less than $100 and you can test any trading strategy or idea imaginable, including trend following, buy and hold ETFs, etc...
The barrier of entry is pretty low and you can develop a great lifestyle business with no customers, employees and investors around that...
Because they do so much work upfront for you. I worked in algorithmic trading for a bit over 10 years but left in 2014. I spent a few months in my spare time building out a trading and backtesting platform, pulling data off of yahoo finance, connecting to Interactive Brokers via their C++ API, etc. It was all very slow and tedious, and during my googling one day, I come across quantopian. I felt like an idiot- they already had free access to morningstar with a decent API around it, tons of other datasets you could subscribe to, and a fully fleshed out backtesting platform with risk measures, pretty graphs, etc.
It would have taken me months-years to get to that point. Shortly after they also included integrations to IB and Robinhood so you could very easily take your ideas and actually live trade them. I immediately stopped development on my trading platform and just started using Q's.
Q could potentially take my algorithm and start using it themselves, though they actually put money into algorithms they think are promising and share the profits with you. That's a small risk IMHO. Hackers and the like... I did this stuff for 10+ years. People WAY over estimate the value of this code- you have an idea, you backtest, hope it works, and then you put it in the wild. I have never seen an algo that is a real guaranteed money maker (aside from arbitrage/hft stuff). What are these hackers going to do when they take a 10% loss one month and don't understand why? Some "signals" can be used in-line fairly generically with other algorithms to enhance their performance, but I will still bet on the guy that knows/understands the algorithms to not lose his shirt, then some hacker dude that likely has no capital to begin with and may not even understand the basics of financial markets.
One thing you seem to overlook is that it is one thing to go buy a bunch of historical data off ebay, but you need to constantly update that if you are going to live trade- Q handles that part as well.
If I was looking to make a business out of this, I would build my own platform. But using it as a research platform to invest my own money, Quantopian is awesome.
Mostly a matter of trust. I trust Quantopian more than I trust some random asshole on eBay purporting to sell accurate historical data. Plus, Quantopian is convenient.
Not everyone can build their own backtesting tool. I played around with quantopian for a little while and did some live trading with quantopian + robinhood and I never would have been able to do that without quantopian
[+] [-] lowpro|8 years ago|reply
So despite the fact that the subject is interesting, I'd consider it a waste of time to try and gain anything but a basic understanding of the industry and how algo trading works.
[+] [-] milcron|8 years ago|reply
[+] [-] dsacco|8 years ago|reply
There is a lot of information asymmetry, and an individual is capable of capturing that without requiring a PhD or the resources of a large firm. That's not to say it's easy per se, but it's not hopeless. It requires special expertise or an unconventional approach.
Otherwise I agree that most people probably shouldn't attempt it (for risk tolerance reasons).
[+] [-] jlg23|8 years ago|reply
MIT does not teach "HFT", those "MIT PHD Quants" are just as untrained in finance as you are. The rest is math, something that requires time to study but not necessarily a certificate. And creativity. I've only seen a single HFT-algo that made me say "wow, that's creative"; all others were just straight implementations of finance "wisdom".
[+] [-] osrec|8 years ago|reply
[+] [-] cocoablazing|8 years ago|reply
An ultra-HFT liquidity provider being profitable has little bearing on the potential profitability of quantitative trading on significantly larger timescales. There isn't sufficient volatility in most products for these actors to eat the lunch of actors who have alpha on trades that can hold for significant periods.
[+] [-] yonkshi|8 years ago|reply
Sure you may not be as profitable as top quant companies, but do you really mind that much?
[+] [-] SirLJ|8 years ago|reply
[+] [-] empath75|8 years ago|reply
[+] [-] etep|8 years ago|reply
[+] [-] neximo64|8 years ago|reply
You've forgotten you can introduce new units of whatever is being traded and quite often is & there are additional complications such as dividends, stock options, etc which makes your oversimplification lacking substance.
[+] [-] bobbington|8 years ago|reply
[deleted]
[+] [-] keithalewis|8 years ago|reply
[+] [-] pmart123|8 years ago|reply
[+] [-] TACIXAT|8 years ago|reply
I think the other goal being that if you create a profitable strategy, you can enter it into competitions, trade with other people's money, and make a profit. I think the platform is a value add, I have an account and use the research notebooks. Some day when I more spare time I'll make some trading algorithms.
[+] [-] vxxzy|8 years ago|reply
[+] [-] hendzen|8 years ago|reply
You should understand the following concepts at a minimum:
- Markowitz portfolio optimization (mean-variance analysis)
- Beta-neutral portfolios (i.e. using MSCI BARRA, sector ETFs or PCA factors, etc)
- Alpha decay
- Time series analysis (autocorrelation, GARCH, ARMA processes)
- Basic price-based signals (momentum, volatility, value, etc) - etc, etc.
[+] [-] ndesaulniers|8 years ago|reply
[+] [-] samolang|8 years ago|reply
[+] [-] JabavuAdams|8 years ago|reply
My takeaway is: it's not about implementing a couple of trading strategies. It's about implementing a pipeline that rapidly allows you to test what-if scenarios.
I might have like 10 ideas a day for strategies. How many of those can I rigorously validate per week? What about variations? I.e. tuning various hyper-parameters? Combinations?
How quickly can I recombine data from various sources into exactly the layout I need to train and test this or that strat?
Me:
1) Build some simple forecasting NNs.
2) Realize I need to be able to generate and test ideas waaay faster. Start working on infrastructure.
3) Get impatient -- get some "bright" ideas, do some manual trading.
4) Make some money on the first day, lose double that on the next two days.
5) Decide to ban myself from manual trading as obviously I'm an addict. Resolve to only do algorithmic trading.
6) Back to coding infrastructure. Get bored ... do some manual trades.
7) After losing around $20k, stop the trading madness, and go back to buying and holding great tech companies.
8) Enjoy my 25% returns.
[+] [-] ndesaulniers|8 years ago|reply
> 7) After losing around $20k
[+] [-] jaredbroad|8 years ago|reply
QuantConnect & LEAN gives you ability to do tick->daily resolutions; for equity, morning-star, future, option, forex and cfd trading - all with a fully open source project which includes samples of data to get you started.
The grunt work is still done in C# so its faster than other full python based backtesting engines. Edit: I'm the Founder @ QC.
[+] [-] rl3|8 years ago|reply
1. Any chance of a Robinhood integration à la Quantopian?
2. How are the architectural revisions[0] coming along?
Also some totally unsolicited feedback:
If I'm being completely honest, I found it difficult to get going with QC. The documentation is decent, but there's not enough to avoid having to review LEAN source right off the bat. The examples also tend to mix helper classes with lower-level functionality, and that can create confusion.
The framework itself feels a bit over-reliant on OOP. Some aspects feel too tightly coupled, others too little. Obviously LEAN has been around for many years now, so architectural baggage is perfectly understandable.
A total rewrite I'm sure isn't feasible, though I'd suggest the following ethos in any case:
a) Design primitive user-accessible data structures with virtually no inbuilt functionality.
b) Build low-level components that operate using those data structures.
c) Build high-level components that compose low-level components.
d) Allow users to author their own components, and to compose components of any type however they see fit.
Pretty sure you're already on that track in a sense, so it's good to see things headed in the right direction. What keeps me from writing a custom framework is the data, the fact QC does a ton of grunt work, and that it's well-tested.
tl;dr Please break apart the monolithic QCAlgorithm class as much as you can! :)
[0] https://www.quantconnect.com/forum/discussion/1816/qc-algori...
[+] [-] SirLJ|8 years ago|reply
[+] [-] tejaswiy|8 years ago|reply
Please do not try to trade actively unless that's your full-time job. Passive investing using index funds is definitely not sexy, but it gets the job done.
[+] [-] rootsudo|8 years ago|reply
[+] [-] gnaritas|8 years ago|reply
Only if you have a significant amount of capital to play with. People looking into active trading are doing it because they don't have the necessary capital to make passive investing meaningful; they're looking for much much larger returns that you can get from an index fund, which necessarily comes with more risk. Traders are gamblers.
[+] [-] TACIXAT|8 years ago|reply
My biggest thing with the Python for Finance books - I know Python, I want to learn finance. All these books are the inverse of that, for people who know finance and want to learn Python. There is a good site for quantitative economics [1] that has tutorials in Python and Julia. I would love a mathematics of finance book that had the examples in Python.
1. https://lectures.quantecon.org/py/
[+] [-] ice109|8 years ago|reply
[+] [-] wtvanhest|8 years ago|reply
[+] [-] Maven911|8 years ago|reply
[+] [-] ncyclopediae|8 years ago|reply
[+] [-] kyleblarson|8 years ago|reply
[+] [-] brobinson|8 years ago|reply
https://en.wikipedia.org/wiki/Walk_forward_optimization
See also: "The Evaluation and Optimization of Trading Strategies" by Robert Pardo
[+] [-] moonforeshot|8 years ago|reply
[+] [-] hagakure0c|8 years ago|reply
[+] [-] NicoJuicy|8 years ago|reply
[+] [-] Veratyr|8 years ago|reply
[+] [-] pfarnsworth|8 years ago|reply
[+] [-] nickthemagicman|8 years ago|reply
Nothing can be predicted because there's way too many confounding factors.
[+] [-] dsacco|8 years ago|reply
Algorithmic trading is very difficult, but it is empirical. Information asymmetry exists in the market, and if you can capture it you profit. I don't know how capacity constrained my strategy is (I think it would be difficult to work this strategy profitably with over $10M depending on how much effort you put into execution; my execution is unsophisticated and I'm working with two orders of magnitude less than that), but it's working.
[+] [-] PaulRobinson|8 years ago|reply
Astrology is story-telling with no scientific grounding.
[+] [-] 21|8 years ago|reply
When filing for its IPO in March 2014, it was disclosed that during five years Virtu Financial made profit 1,277 out of 1,278 days, losing money just one day.
[+] [-] fenwick67|8 years ago|reply
[+] [-] SirLJ|8 years ago|reply
They are your competitor and who will prevent a disgruntle employee or a hacker to steel your successful trading strategy?
Just buy some data from eBay, you can get 20 years of historical stock market data for less than $100 and you can test any trading strategy or idea imaginable, including trend following, buy and hold ETFs, etc...
The barrier of entry is pretty low and you can develop a great lifestyle business with no customers, employees and investors around that...
[+] [-] kevstev|8 years ago|reply
It would have taken me months-years to get to that point. Shortly after they also included integrations to IB and Robinhood so you could very easily take your ideas and actually live trade them. I immediately stopped development on my trading platform and just started using Q's.
Q could potentially take my algorithm and start using it themselves, though they actually put money into algorithms they think are promising and share the profits with you. That's a small risk IMHO. Hackers and the like... I did this stuff for 10+ years. People WAY over estimate the value of this code- you have an idea, you backtest, hope it works, and then you put it in the wild. I have never seen an algo that is a real guaranteed money maker (aside from arbitrage/hft stuff). What are these hackers going to do when they take a 10% loss one month and don't understand why? Some "signals" can be used in-line fairly generically with other algorithms to enhance their performance, but I will still bet on the guy that knows/understands the algorithms to not lose his shirt, then some hacker dude that likely has no capital to begin with and may not even understand the basics of financial markets.
One thing you seem to overlook is that it is one thing to go buy a bunch of historical data off ebay, but you need to constantly update that if you are going to live trade- Q handles that part as well.
If I was looking to make a business out of this, I would build my own platform. But using it as a research platform to invest my own money, Quantopian is awesome.
[+] [-] mi100hael|8 years ago|reply
[+] [-] Blackstone4|8 years ago|reply
[+] [-] dglass|8 years ago|reply