Here is the working paper [1] if anyone is more interested in the details. The gist of it is that there are two ways to get insider transaction reports from EDGAR. One way is to scrape the EDGAR website (polling), the other is to sign up for a push service [2] where the filings are sent to you as soon as they hit EDGAR. You have to pay for the latter service. The paper calls the scraping solution "public" even though both solutions are open to the public (though you have to pay for the push service). According to the paper, it turns out (maybe not so shockingly) that the push service is better than the scraping/polling solution about half the time. The paper is written by economists and not technologists, and so there is no mention of CDNs or web caches a.k.a. the types of things most developers would suspect as being the most obvious source of the discrepancy.
Ah thanks for the link to the working paper, I had been looking for it. This jumps out from the footnote on page 7:
"As far as we know, this time is not available on any publicly available database. We initially obtained these times using real time “scrapes” of the SEC EDGAR site. We subsequently used a collection of these times made available by the Tier 1 subscriber.
Given that this entity’s business model depends, at least in part, on obtaining and disseminating these filings in the most timely manner possible, they have strong incentives to collect accurate information about when filings become available on the SEC website."
Basically, clock skew is not accounted for at all and timings are potentially done from disparate and non-common observation points. It's quite possible that the delays they observe are strictly due to noise.
There are no public prices and you need to email some company for more details. With most businesses this means you contact a salesperson who will play an extended game of "how much you got?" and the pricing AND the level of service provided will vary widely depending upon your negotiating power.
The dearth of information about this 'product' sold by a 3rd party suggests something shady is probably going on.
>The paper is written by economists and not technologists, and so there is no mention of CDNs or web caches
Caching is a pretty poor excuse. Caches can be invalidated. And CDN basically means caching by another name.
It makes much more sense (both economically and technically) that they simply crippled the free feed. Especially since it is provided by a private third party who makes bank on the people who purchase the premium edition.
I'm wondering if data could be distributed ahead of time in an encrypted fashion and then only a key needs to be published/pushed, which is a far more trivial thing to get out to many people simultaneously.
I think it's very unfortunate that they offer two different data sources which provide an unfair advantage to paying subscribers. Then again I don't think there's a single serious investor out there, who attempts to gain an advantage by scraping the crummy EDGAR site. Regardless I guess this is just what you get when you outsource a public service to a private company like EDGAR Online.
"Is this bad? I mean, look, one: It doesn't matter at all. We'll get to that. But, two: Sure, it's bad! It's symbolically stupid. The point of the Form 4 is that the SEC wants everyone to know when corporate insiders buy or sell stock, so that all the little investors can compete on a level playing field. As a goal, this has its problems, but it's a goal. For the SEC itself to give this disclosure to the little investors after professionals get it is not a great look. "
...
"But when I say it doesn't matter at all, I mean, it does not matter at all. The idea here is that subscribing to a news service or data terminal gives "professional traders an edge over mom-and-pop investors." The article really says that. Now, this seems pretty obvious. Most of the time, professionals using professional tools will be better at doing things than amateurs using amateur tools. There are very few fields of human endeavor where professionals do not have an edge over moms and pops. But investing might come closest! You and your mom and your pop can just index! It is great, you will beat the majority of professional fund managers every year."
...
"if you are a mom-or-pop investor, and you are day-trading the stock of a $950 million specialty chemicals company based on your instant reaction to news that a non-executive director has bought $194,000 worth of stock,5 then you have already lost all your money. Nothing that I, or the SEC, or Eric Schneiderman, could ever do will help you. You are doomed."
The SEC is a government agency. It should not be favoring some citizens over others, and the SEC should not be giving market advantages to those it regulates.
(b) Whether the delay was 0 or 100 seconds, the computer consuming the feed will always be faster than a human.
(c) The details of the study are murky and unknown. In particular, the data collection aspect is suspect given it sounds like the researchers received third party data not directly collected by themselves. How were the timestamps handled? Were various caching issues properly accounted for? (The SEC website is served up via Akamai, for example).
Anyway, it's great a populist topic these days. Evil HFT always beating you to the punch, etc. But in reality, it doesn't matter because they are always faster than you. I suggest you read this post by well known finance/trading blogger Kid Dynamite: http://kiddynamitesworld.com/someone-will-always-have-the-da...
Yeah, I'm not sure why it should be assumed the web works the same as a dedicated feed. If I refresh a webpage that has "realtime" market data, I still had to wait for HTTP(S), the web server, network latency, my web browser, etc. to render the number. If I had a dedicated line and realtime API I would undoubtedly get the number relatively faster than anyone getting it from scraping a web page.
If you know that someone else has an edge, you can plan for it and you know how much risk you're taking on. The stock market is heavily regulated to reduce risk in trading. The scandal isn't necessarily that some people get the info early (although that plays better in the news), but that it's a secret.
I'm not sure this is really much of a scandal. HFT / algo-trading firms are generally trying to make a buck on market momentum, not fundamentals.
Investors who take time to actually read and interpret financial statements, if they're good, can observe-orient-decide-act quickly, but 100 seconds does not provide much of an advantage.
To the extent that there is a surprise in the Q or K, it's usually divulged ahead of time in a revised earnings guidance by the company itself (albeit this doesn't always happen when the surprise is to the upside).
Either way, the equity hedge funds that care about the details of financial statements don't get their edge from a one-minute head start on reading these statements; they get it from paying shady "industry consultants" to feed them scuttlebutt.
HFT / algo-trading firms are NOT generally trying to make a buck on market momentum. They're generally trying to make a buck by market making. They're buck (or penny really) comes from the bid/ask spread.
I don't think it would be that hard to have a computer program scrape a form, compare it to industry predictions and take a position in well under a second.
There was a scandal last year when it was discovered that Thomson Reuters paid the University of Michigan for early access to their market reports. http://www.cnbc.com/id/100809395 That was discovered when they traded 2 seconds before everyone else. You'd think the SEC would have known since then that this is detectable.
Well this is exactly how I managed to get a lot of free food and cruft back in the days when I was an undergrad at MIT. There was a mailing list for this stuff called Reuse, but rather than subscribe to the mailing list (whose servers were slow), I subscribed instead to the Zephyr class of the corresponding discuss archive and pulled the message out of the archive -- often 30-60 seconds before the e-mails would show up in peoples' inboxes. Too bad it doesn't work anymore after they switched to Mailman and killed the discuss archive.
This or something similar was on 60 Minutes a few months ago it was Canadian head trader at RBC Brad Katsuyama's work got the attention of Michael Lewis.
When Katsuyama placed large orders only some were filled and the rest later at a high price. Exchanges closer to the stock market got trade data faster and could outbid people.
In the end Katsuyama started his own exchange IEX and purposely delays their trades using loops of thousands of feet of fibre optic cable (shown on 60 Minutes) to hide from the faster traders.
A lot of the big traders even Warren Buffet included dismiss it as sour grapes.
> This or something similar was on 60 Minutes a few months ago
In that it may involve HFT, it is similar. But otherwise, no. RBC's problem was in failing to handle latency arbitrage when attempting to execute against all 13-14 exchanges in parallel. Which was solved by Thor, not by IEX. And IEX is a dark pool, not an exchange (quotes are not protected, nor even visible!)
The news here is that SEC is potentially underhandedly running a news service business for earnings traders. I don't think it's much news, but it is quite different from the "Flash Boys" discussion.
I'm not sure about the macro-level benefits of this kind of high speed trading. Yes, there are winners and there will be losers. The common-folk won't be able to win consistently with high-speed trading strategies. But, FOREX has been like this for years already.
I wonder if the root of the issue could be addressed by making a requirement that valid stock transactions can only happen every 10 seconds. Transactions could be initiated any time, but the price of the stock would be finalized on transaction every N seconds. The intent to purchase (beginning of the transaction) could specify how many stocks to purchase and what is the maximum amount of money that could be spent for the transaction. A single service would need to be the authority of all transactions.
I do not know much about how the stock market works, but I'd think that addressing the unfairness of computerized trading would not be an impossible thing to address.
Interestingly, the exchanges have been under fire from the SEC in regards to the dissemination of marketdata. Are the exchanges sending out data from their prop feeds at the same time it is submitted to the SIP?
Well if it is measured at the 1-second or 1-millisecond level, yes the exchanges are in compliance. At the 100-microsecond, 10-microsecond and nanosecond level, perhaps the exchange is in compliance at the 50th percentile.
It will be interesting to see if the pressure on the exchanges changes course as they empathize with the same type of dissemination problem.
Exchanges do send data to both the SIP and direct data feeds at the same time. The problem is that the SIP has an intrinsic disadvantage due to the architecture and technology used that guarantee SIP market data will always be slower than direct market data.
I'm hopeful that, in time, this will be fixed. Not so much because I believe the latency induced by the flawed SIP architecture is material to SIP subscribers, but instead for 2 reasons:
1. The very same architecture that adds latency makes the SIP a SPOF in our market system and as the NASDAQ Tape C outage showed, it can really suck when the SIP doesn't work;
2. The PERCEPTION of unfairness is much more harmful than any actual harm done due to the SIP/direct latency delta. Fixing the SIP can directly correct the source of the perception of unfairness and bring some credibility to the market place and its governance.
>Well if it is measured at the 1-second or 1-millisecond level, yes the exchanges are in compliance. At the 100-microsecond, 10-microsecond and nanosecond level, perhaps the exchange is in compliance at the 50th percentile.
And once you hit the <1 millisecond level, it's hard to even get reliable measurements, which makes compliance for "same time delivery" really really freaking hard.
Keep in mind, when you're talking about the nanosecond level, you're at the point where the length of cabling between the systems matters.
Sure, there's a lot of new tech coming out, especially with using GPS to synchronize clocks, but it's still a major issue.
For any amount of time less than it would take a human being to make a decision and take action, it doesn't really matter the latency. Even if the website and newsfeed release were guaranteed simultaneous, the mom & pop investor typically doesn't have a computerized trading strategy, let alone one running in a colocation facility with a single switch hop between them and the exchanges.
Humans make the decisions, but they make them in advance. Machines are programmed to parse the feeds and trade based on their content. This has almost nothing to do with retail investing. It has to do with big players paying for the privilege to front-run the rest of the market.
'Fair' release of market-moving information would be a good place to use true 'broadcast' technology: radio waves, rather than broadcast-like network pushes.
Potentially, also, an encrypted full-length report could be pre-released. Only after enough time for the ciphertext to be widely replicated to all interested parties would the short decryption key be radio-broadcast.
This seems like an easy problem to fix. There should be a quiet period between market close and after-market open when all documents are released, so that everyone has a chance to see and process the documents.
This is fixing the wrong problem. The issue is not the rate at which people are getting information; it is the rate at which people are using the information which creates the volatility.
My solution is as follows. When someone places a market order, the order is not cancelable and his funds or security goes into escrow. Then he must wait for a amount of time, selected at random from some distribution by the market. For example, he might have to wait 5 minutes; or wait 2 hours. At that point, the transaction occurs. The market will clear all transactions before it closes. The idea is to significantly reduce the advantage people might gain from rapid transactions. To discourage breaking transactions into many microtransactions for purposes of gaming the distribution, we might also introduce a small per-transaction tax.
Yea, I don't really get this. You don't even need the markets to close. You just need to announce ahead of time when such important documents will be released. (I assume this is already done rather than releasing them as a surprise.) Then, people trading slowly will simply refrain from trading their stocks until they have read the report. Having this info 10 seconds earlier doesn't help the fast trader if no one is willing to trade with them.
[+] [-] minimax|11 years ago|reply
1. http://online.wsj.com/public/resources/documents/SECDissemin...
2. http://www.sec.gov/info/edgar/ednews/dissemin.htm
edit: There is a WSJ article suggesting the price for the push feed is around $1500 / month.
http://online.wsj.com/articles/fast-traders-are-getting-data...
[+] [-] lrm242|11 years ago|reply
"As far as we know, this time is not available on any publicly available database. We initially obtained these times using real time “scrapes” of the SEC EDGAR site. We subsequently used a collection of these times made available by the Tier 1 subscriber. Given that this entity’s business model depends, at least in part, on obtaining and disseminating these filings in the most timely manner possible, they have strong incentives to collect accurate information about when filings become available on the SEC website."
Basically, clock skew is not accounted for at all and timings are potentially done from disparate and non-common observation points. It's quite possible that the delays they observe are strictly due to noise.
[+] [-] crdoconnor|11 years ago|reply
There are no public prices and you need to email some company for more details. With most businesses this means you contact a salesperson who will play an extended game of "how much you got?" and the pricing AND the level of service provided will vary widely depending upon your negotiating power.
The dearth of information about this 'product' sold by a 3rd party suggests something shady is probably going on.
>The paper is written by economists and not technologists, and so there is no mention of CDNs or web caches
Caching is a pretty poor excuse. Caches can be invalidated. And CDN basically means caching by another name.
It makes much more sense (both economically and technically) that they simply crippled the free feed. Especially since it is provided by a private third party who makes bank on the people who purchase the premium edition.
[+] [-] malandrew|11 years ago|reply
[+] [-] jcfrei|11 years ago|reply
[+] [-] philrapo|11 years ago|reply
http://www.bloombergview.com/articles/2014-10-29/high-speed-...
"Is this bad? I mean, look, one: It doesn't matter at all. We'll get to that. But, two: Sure, it's bad! It's symbolically stupid. The point of the Form 4 is that the SEC wants everyone to know when corporate insiders buy or sell stock, so that all the little investors can compete on a level playing field. As a goal, this has its problems, but it's a goal. For the SEC itself to give this disclosure to the little investors after professionals get it is not a great look. " ...
"But when I say it doesn't matter at all, I mean, it does not matter at all. The idea here is that subscribing to a news service or data terminal gives "professional traders an edge over mom-and-pop investors." The article really says that. Now, this seems pretty obvious. Most of the time, professionals using professional tools will be better at doing things than amateurs using amateur tools. There are very few fields of human endeavor where professionals do not have an edge over moms and pops. But investing might come closest! You and your mom and your pop can just index! It is great, you will beat the majority of professional fund managers every year." ...
"if you are a mom-or-pop investor, and you are day-trading the stock of a $950 million specialty chemicals company based on your instant reaction to news that a non-executive director has bought $194,000 worth of stock,5 then you have already lost all your money. Nothing that I, or the SEC, or Eric Schneiderman, could ever do will help you. You are doomed."
[+] [-] hackuser|11 years ago|reply
[+] [-] lrm242|11 years ago|reply
(a) The SEC feed can be subscribed to by anyone.
(b) Whether the delay was 0 or 100 seconds, the computer consuming the feed will always be faster than a human.
(c) The details of the study are murky and unknown. In particular, the data collection aspect is suspect given it sounds like the researchers received third party data not directly collected by themselves. How were the timestamps handled? Were various caching issues properly accounted for? (The SEC website is served up via Akamai, for example).
Anyway, it's great a populist topic these days. Evil HFT always beating you to the punch, etc. But in reality, it doesn't matter because they are always faster than you. I suggest you read this post by well known finance/trading blogger Kid Dynamite: http://kiddynamitesworld.com/someone-will-always-have-the-da...
[+] [-] apaprocki|11 years ago|reply
[+] [-] sp332|11 years ago|reply
[+] [-] leroy_masochist|11 years ago|reply
Investors who take time to actually read and interpret financial statements, if they're good, can observe-orient-decide-act quickly, but 100 seconds does not provide much of an advantage.
To the extent that there is a surprise in the Q or K, it's usually divulged ahead of time in a revised earnings guidance by the company itself (albeit this doesn't always happen when the surprise is to the upside).
Either way, the equity hedge funds that care about the details of financial statements don't get their edge from a one-minute head start on reading these statements; they get it from paying shady "industry consultants" to feed them scuttlebutt.
[+] [-] harryh|11 years ago|reply
[+] [-] sah88|11 years ago|reply
[+] [-] sp332|11 years ago|reply
Old edit removed for being wrong :p
[+] [-] maxerickson|11 years ago|reply
The elite clients paid for access to a conference call 5 minutes ahead of the public announcement of the data.
The super duper elite clients paid for access to an electronic feed that provided the data 2 seconds before that conference call began.
[+] [-] coldcode|11 years ago|reply
What the SEC could do and what they actually do seem to have little in common.
[+] [-] dheera|11 years ago|reply
[+] [-] dghughes|11 years ago|reply
When Katsuyama placed large orders only some were filled and the rest later at a high price. Exchanges closer to the stock market got trade data faster and could outbid people.
In the end Katsuyama started his own exchange IEX and purposely delays their trades using loops of thousands of feet of fibre optic cable (shown on 60 Minutes) to hide from the faster traders.
A lot of the big traders even Warren Buffet included dismiss it as sour grapes.
http://www.cbc.ca/news/business/canadian-brad-katsuyama-in-s...
[+] [-] ycombobreaker|11 years ago|reply
In that it may involve HFT, it is similar. But otherwise, no. RBC's problem was in failing to handle latency arbitrage when attempting to execute against all 13-14 exchanges in parallel. Which was solved by Thor, not by IEX. And IEX is a dark pool, not an exchange (quotes are not protected, nor even visible!)
The news here is that SEC is potentially underhandedly running a news service business for earnings traders. I don't think it's much news, but it is quite different from the "Flash Boys" discussion.
[+] [-] ececconi|11 years ago|reply
This made me think of the great series here: http://www.chrisstucchio.com/blog/2012/hft_apology.html
[+] [-] unknown|11 years ago|reply
[deleted]
[+] [-] mrinterweb|11 years ago|reply
I do not know much about how the stock market works, but I'd think that addressing the unfairness of computerized trading would not be an impossible thing to address.
[+] [-] itchyouch|11 years ago|reply
Well if it is measured at the 1-second or 1-millisecond level, yes the exchanges are in compliance. At the 100-microsecond, 10-microsecond and nanosecond level, perhaps the exchange is in compliance at the 50th percentile.
It will be interesting to see if the pressure on the exchanges changes course as they empathize with the same type of dissemination problem.
[+] [-] lrm242|11 years ago|reply
I'm hopeful that, in time, this will be fixed. Not so much because I believe the latency induced by the flawed SIP architecture is material to SIP subscribers, but instead for 2 reasons:
1. The very same architecture that adds latency makes the SIP a SPOF in our market system and as the NASDAQ Tape C outage showed, it can really suck when the SIP doesn't work;
2. The PERCEPTION of unfairness is much more harmful than any actual harm done due to the SIP/direct latency delta. Fixing the SIP can directly correct the source of the perception of unfairness and bring some credibility to the market place and its governance.
[+] [-] cube13|11 years ago|reply
And once you hit the <1 millisecond level, it's hard to even get reliable measurements, which makes compliance for "same time delivery" really really freaking hard.
Keep in mind, when you're talking about the nanosecond level, you're at the point where the length of cabling between the systems matters.
Sure, there's a lot of new tech coming out, especially with using GPS to synchronize clocks, but it's still a major issue.
[+] [-] omellet|11 years ago|reply
[+] [-] fnordfnordfnord|11 years ago|reply
[+] [-] gojomo|11 years ago|reply
Potentially, also, an encrypted full-length report could be pre-released. Only after enough time for the ciphertext to be widely replicated to all interested parties would the short decryption key be radio-broadcast.
[+] [-] unknown|11 years ago|reply
[deleted]
[+] [-] jedberg|11 years ago|reply
[+] [-] SeanLuke|11 years ago|reply
My solution is as follows. When someone places a market order, the order is not cancelable and his funds or security goes into escrow. Then he must wait for a amount of time, selected at random from some distribution by the market. For example, he might have to wait 5 minutes; or wait 2 hours. At that point, the transaction occurs. The market will clear all transactions before it closes. The idea is to significantly reduce the advantage people might gain from rapid transactions. To discourage breaking transactions into many microtransactions for purposes of gaming the distribution, we might also introduce a small per-transaction tax.
[+] [-] jessriedel|11 years ago|reply
[+] [-] Link-|11 years ago|reply
[+] [-] ashleyw|11 years ago|reply
[+] [-] unknown|11 years ago|reply
[deleted]