> It showed a direct link between the West Chicago tower and another tower right by the Chicago Mercantile Exchange!
It's pointing at the top of 10 South Canal, one of downtown Chicago's main tombstones for data communications. The building has 3-6 foot concrete walls, no windows for the first 23 stories and was reportedly designed to withstand an overhead nuclear blast. The standing joke used to be that if Chicago were nuked you'd still get a phone bill because of that place.
There's a lot of underground fiber in that area too. Including 24-count bundles mounted near the ceiling of the old underground tunnel delivery system -- the one that accidentally flooded during the 90's. They were installed by Metropolitan Cable back in the late eighties/early nineties during the "politically connected" fiber gold rush to replace dedicated copper with light.
It would not be speculation to think there's fiber running directly from the MERC to 10 South Canal or that the MERC's location near 10 South is a coincidence.
The line-of-sight out to Aurora is interesting too. There are high spots out there with tower clusters reminiscent of East Texas in the 1930's. I believe there are backup trading centers out there as well built after 9/11.
i am not sure i would agree that this has been discussed before. this is not simply about use of RF (microwave, mmwave) for HFT. as i understand it, that kind of thing typically involves COTS radios and conventional dish antennas.
this instance appears to involve longer-range links and SDRs (in this case, the ettus USRPs).
A microwave radio setup to the nearest populated area gave my rural-living parents modern internet a few years ago. It’s not the same scale but the same idea! It’s far and away better than satellite.
Interesting... I actually know someone working on a project similar to this (who works at an HFT firm) - if not actually this.
His team was simulating and testing how various weather conditions could affect trade placement times. For instance, does a high-pressure system, a heat wave, a blizzard, etc. impact the trade placement times, if so how does that impact the algorithms.
He works out of Chicago and I've heard of similar things before, so this isn't unknown to people in the area.
If this article was an intro to a fiction novel, I’d be interested in reading the entire thing. Some sort of HFT heist with a good mix of factual engineering and finance.
This is a wonderful article for so many reasons. Outlines resources for doing this yourself, uses industry knowledge to outline hypotheses and then tests them with data, and reveals a really interesting possibility for market communication. Also, a bit of local Chicago geography :) An enjoyable read!
Edit: the “is this a troll” comment makes this even more enjoyable for some reason. Now it’s an even bigger treasure hunt.
Would love to know what licensing regime / category they are using for this. It definitely isn't ham, because that's non-commercial only. The fact that log period antennas are in use is interesting too - low gain but directional and useful across a broad frequency range - maybe some spread spectrum technology?
Probably licensed on several bands, as propagation conditions change quickly. They’d need to hop around, if not simultaneously transmitting on multiple bands.
All along everyone thought that numbers stations were how governments send messages to their spooks, in reality they were just poor bastards holed up in tiny shacks manually reading the tape for the S&P e-mini...
One has to wonder if this cutthroat competition will ever turn aggressive. Someone with a SDR and a bit of knowhow could shutdown/jam these systems very easily. The apparatus is in public view and they don't seem to be guarding their trash. A well-timed attack on a couple of these towers would create a few seconds of instability on the markets, enough time for another HFT platform to take advantage.
A blast from a 1-watt transmitter parked across the street would overwhelm any signal coming from across an ocean. Try calling the cops about to complain that your antenna reception is being interfered with. The days of cops hunting rogue radio stations are long gone.
I guess low bandwidth is no problem. You only need to send a stream of price changes of the most common instruments (currencies, stocks). And only if there are meaningful movements. I guess a few kbit/s can already provide a lot of value.
How fast would such a connection be? Is there an easy way to calculate the distance the signal takes compared to great circle line?
They are probably also monitoring HF propagation paths, as there can be ground wave and sky wave, with varying dispersion.
The X300 has some latency too. I’ll be they are doing the majority of processing directly on the FPGA to avoid SDR latency. The X310 has a larger FPGA, and would be tell-tale.
Bring on the comms with neutrinos or entangled particles.
The Author's comments refer to an internet forum thread[0] in German which discusses a German news article from this April.[1]However, it turns out this article is an April Fools text.
Just digging deeper into the whole blog you can find something that can only be described as a fanfiction representation of "Flashboys", but in the context of this "Shortwave trading".[2]
Is this whole thing a troll? Is this real? What on earth is this?
Long-time "sniper in mahwah" follower here. The author is without the slightest doubt an elite tech reporter with in-depth understanding of RF tech. He always provides an overwhelming body of evidence to back his writing.
The linked article [1] appears to be built around a word-play, where "Shortwave"-trading and "High frequency"-trading are ambiguously used as "Hochfrequenz"handel
From what I understand (and I'm no expert in HFT or electric engineering), this kind of sub-second trading is done in very specific corridors: Chicago-NY, NY-London, London-Frankfurt, London-Paris, Paris-Frankfurt. I wonder if the same applies in Asia, with Singapore, HK, Tokyo, Sidney...
I'm not even remotely involved with the HFT world, but I can't think of any good reason why they'd want to receive "shortwave" signals from Europe in Chicago.
Short-haul microwave links for low latency and high bandwidth, yes, but at shortwave frequencies you are literally talking about BITS per second. TFA says "Think dialup speeds" but this is lower than even dialup!
They certainly aren't receiving any meaningful amount of data (which I presume you need for HFT) with that shortwave receiver.
The crucial information is up and down ticks of the ES-mini futures at the Chicago Mercantile Exchange. These lead the S&P500 (SPY) index funding in NYC (by 4 milliseconds), and the ES-mini ticks up and down about once per second. This few bits of information carries huge value since much of the world's equity market responds to it. See https://arxiv.org/abs/1302.5966
As a Ham I am certainly intrigued by this posting.
There are a number of obstacles I can see to effectively using shortwave for financial services. Propagation conditions vary by time of day, solar activity and other exogenous factors. Another post hypothesized a 100 Kw transmitter power but even without running the numbers it seems like the FCCs Maximum Permissible Exposure Limits (MPE) would dictate a larger exclusion zone than the author encountered, however proper antenna modeling would be needed to be completely accurate. Another confounding factor to potential efficacy of the site is the surrounding terrain. The photos show nearby structures and trees which are less conducive to effective take off angles and propagation associated with the Yagi type antennas depicted. Most hardcore Ham sites with big towers have well groomed earth (or even better, water) around them. Additionally, the data would obviously have be encrypted and transmitted with extremely robust error correction.
It would be fun to grab a couple of boxes from AOR and see what is coming off those antennas.
From the McKay brothers' website: "It's better to be fast 99% of the time than slow 99.999% of the time"
- Bob Meade and Stéphane Tyč, Co-Founders
If your trading strategy is simply arbitraging Chicago/London, it's probably ok if your link is even 50% reliable, if for that 50% of the time, it's significantly faster than your competition's data. London is going to get the information 10ms later anyhow, so if a packet drops, you can still execute the offsetting transaction, just at the same time as everyone else.
This was covered pretty extensively by Crain's Chicago Business. Some companies are so obsessed with this sort of thing that they'll pay half a million dollars to move a receive site across the street. They measure results in picoseconds. Too much pressure for me.
I worked on a project in several states for IBM that handled traffic shaping and detection on highways and onramps. This system was used to identify gridlock and adjust lights at offramps or give people an ETA until theyre downtown. Most adjacent cities were staunchly NIMBY and didnt care to have the state government trenching cable to and fro for traffic control, so the data ended up hopping from town to town on microwave subcontracted from the cellular company that handled the large electric billboards on the highway.
the benefit of this was also the ability to route signal control data to large intersections without having to shut them down to dig cable trenches from the IBM network.
I recall seeing I think on 60 Minutes HFT that wanted worse latency but I can't recall why. They have massive spools of fiber optics using them as a "resistor" (my term) to slow down the transmission.
It seems to be the opposite of this but for the life of me I can't remember the advantage of doing it.
I think you're referring to IEX, an exchange that adds a delay with those spools of fiber optics. According to wiki, "350 microsecond delay adds a round-trip delay of 0.0007 seconds and is designed to negate the certain speed advantages utilized by some high-frequency traders".
The reason is that "the market" is made up multiple exchanges and to make a trade you need to do it on a specific exchange, but stocks are spread across all of the exchanges so if you need to make a big trade you have to send the trade order to several exchanges at once.
The problem is that HFT firms will see the trade hit the closest exchange (in terms of network latency), and then rush out to buy out all of the remaining shares of that stock at that price on the other exchanges (beating you because they HFT better) and relisting them at a higher price.
HFT firms will tell you that this is perfectly acceptable behavior because if you then buy the stock from them that the original person simply wasn't charging enough for it. They will also tell you that it improves liquidity, but that doesn't make sense either. It's not like they're sitting on the shares keeping the markets moving. All it does is add a tax to trades.
IEX delays the transactions just enough that the nanosecond advantages that HFT enjoys can't be used to front run trades.
It's been a few years since I was involved with any of this, but even for higher-bandwidth microwave networks you want to squeeze every last bit out of the protocol.
I'd guess in this instance that they'll skip both the TCP and IP layers completely and essentially be sending minimally encapsulated payloads out almost raw. Minimally encapsulated at this level might actually mean no encapsulation. Maybe some error-detection/CRC type stuff as these links tend to be super lossy.
You probably want to combine in a single raw packet the data, compressed with a prediction codec with the error detection code. Since you also have a lower-latency high-speed link (the transatlantic cable), you can continuously update the probabilistic model used by the compression to squeeze the last bit.
I haven't worked with microwave feeds, but from the war stories I have heard, not even IP is used but data is sent using whatever low level protocol (Ethernet or something similar) is used by the transport.
In fact the payload might be empty or mostly empty and data can be encoded in protocol fields (for example encoding symbol names in address fields as the link is otherwise point to point).
[+] [-] johnohara|8 years ago|reply
It's pointing at the top of 10 South Canal, one of downtown Chicago's main tombstones for data communications. The building has 3-6 foot concrete walls, no windows for the first 23 stories and was reportedly designed to withstand an overhead nuclear blast. The standing joke used to be that if Chicago were nuked you'd still get a phone bill because of that place.
There's a lot of underground fiber in that area too. Including 24-count bundles mounted near the ceiling of the old underground tunnel delivery system -- the one that accidentally flooded during the 90's. They were installed by Metropolitan Cable back in the late eighties/early nineties during the "politically connected" fiber gold rush to replace dedicated copper with light.
It would not be speculation to think there's fiber running directly from the MERC to 10 South Canal or that the MERC's location near 10 South is a coincidence.
The line-of-sight out to Aurora is interesting too. There are high spots out there with tower clusters reminiscent of East Texas in the 1930's. I believe there are backup trading centers out there as well built after 9/11.
[+] [-] retzkek|8 years ago|reply
[+] [-] johnohara|8 years ago|reply
[+] [-] discussedbefore|8 years ago|reply
HFT in My Backyard https://news.ycombinator.com/item?id=8354278 (2014)
[+] [-] aoki|8 years ago|reply
this instance appears to involve longer-range links and SDRs (in this case, the ettus USRPs).
[+] [-] cjsawyer|8 years ago|reply
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] lettergram|8 years ago|reply
His team was simulating and testing how various weather conditions could affect trade placement times. For instance, does a high-pressure system, a heat wave, a blizzard, etc. impact the trade placement times, if so how does that impact the algorithms.
He works out of Chicago and I've heard of similar things before, so this isn't unknown to people in the area.
[+] [-] sniperinmahwah|8 years ago|reply
[+] [-] social_quotient|8 years ago|reply
[+] [-] yangcanvas|8 years ago|reply
[+] [-] sailfast|8 years ago|reply
Edit: the “is this a troll” comment makes this even more enjoyable for some reason. Now it’s an even bigger treasure hunt.
[+] [-] tomfanning|8 years ago|reply
[+] [-] Jommi|8 years ago|reply
[+] [-] madengr|8 years ago|reply
[+] [-] a-dub|8 years ago|reply
[+] [-] sandworm101|8 years ago|reply
A blast from a 1-watt transmitter parked across the street would overwhelm any signal coming from across an ocean. Try calling the cops about to complain that your antenna reception is being interfered with. The days of cops hunting rogue radio stations are long gone.
[+] [-] jandrese|8 years ago|reply
[+] [-] dx034|8 years ago|reply
How fast would such a connection be? Is there an easy way to calculate the distance the signal takes compared to great circle line?
[+] [-] coob|8 years ago|reply
Do all of the processing at the source and just send the final instruction?
(honest question I have no idea how this works)
[+] [-] edf13|8 years ago|reply
[+] [-] madengr|8 years ago|reply
The X300 has some latency too. I’ll be they are doing the majority of processing directly on the FPGA to avoid SDR latency. The X310 has a larger FPGA, and would be tell-tale.
Bring on the comms with neutrinos or entangled particles.
[+] [-] andai|8 years ago|reply
[+] [-] Jommi|8 years ago|reply
The Author's comments refer to an internet forum thread[0] in German which discusses a German news article from this April.[1]However, it turns out this article is an April Fools text.
Just digging deeper into the whole blog you can find something that can only be described as a fanfiction representation of "Flashboys", but in the context of this "Shortwave trading".[2]
Is this whole thing a troll? Is this real? What on earth is this?
[0]http://funkbasis.de/viewtopic.php?f=39&t=45900
[1]https://www.heise.de/newsticker/meldung/Boersenhandel-besche...
[2]https://sniperinmahwah.wordpress.com/2017/06/26/network-effe...
[+] [-] avip|8 years ago|reply
[+] [-] nielsole|8 years ago|reply
[+] [-] Jommi|8 years ago|reply
[+] [-] baxtr|8 years ago|reply
[+] [-] Ftuuky|8 years ago|reply
[+] [-] jlgaddis|8 years ago|reply
Short-haul microwave links for low latency and high bandwidth, yes, but at shortwave frequencies you are literally talking about BITS per second. TFA says "Think dialup speeds" but this is lower than even dialup!
They certainly aren't receiving any meaningful amount of data (which I presume you need for HFT) with that shortwave receiver.
[+] [-] IndexicalDemon|8 years ago|reply
[+] [-] mathgenius|8 years ago|reply
[+] [-] slr555|8 years ago|reply
There are a number of obstacles I can see to effectively using shortwave for financial services. Propagation conditions vary by time of day, solar activity and other exogenous factors. Another post hypothesized a 100 Kw transmitter power but even without running the numbers it seems like the FCCs Maximum Permissible Exposure Limits (MPE) would dictate a larger exclusion zone than the author encountered, however proper antenna modeling would be needed to be completely accurate. Another confounding factor to potential efficacy of the site is the surrounding terrain. The photos show nearby structures and trees which are less conducive to effective take off angles and propagation associated with the Yagi type antennas depicted. Most hardcore Ham sites with big towers have well groomed earth (or even better, water) around them. Additionally, the data would obviously have be encrypted and transmitted with extremely robust error correction.
It would be fun to grab a couple of boxes from AOR and see what is coming off those antennas.
[+] [-] cameldrv|8 years ago|reply
If your trading strategy is simply arbitraging Chicago/London, it's probably ok if your link is even 50% reliable, if for that 50% of the time, it's significantly faster than your competition's data. London is going to get the information 10ms later anyhow, so if a packet drops, you can still execute the offsetting transaction, just at the same time as everyone else.
[+] [-] reaperducer|8 years ago|reply
[+] [-] nimbius|8 years ago|reply
I worked on a project in several states for IBM that handled traffic shaping and detection on highways and onramps. This system was used to identify gridlock and adjust lights at offramps or give people an ETA until theyre downtown. Most adjacent cities were staunchly NIMBY and didnt care to have the state government trenching cable to and fro for traffic control, so the data ended up hopping from town to town on microwave subcontracted from the cellular company that handled the large electric billboards on the highway.
the benefit of this was also the ability to route signal control data to large intersections without having to shut them down to dig cable trenches from the IBM network.
[+] [-] jlgaddis|8 years ago|reply
[+] [-] dghughes|8 years ago|reply
It seems to be the opposite of this but for the life of me I can't remember the advantage of doing it.
Found it: https://www.theatlantic.com/business/archive/2016/06/iex-app...
[+] [-] Ftuuky|8 years ago|reply
https://en.wikipedia.org/wiki/IEX Edit: didn't saw your edit.
[+] [-] jandrese|8 years ago|reply
The problem is that HFT firms will see the trade hit the closest exchange (in terms of network latency), and then rush out to buy out all of the remaining shares of that stock at that price on the other exchanges (beating you because they HFT better) and relisting them at a higher price.
HFT firms will tell you that this is perfectly acceptable behavior because if you then buy the stock from them that the original person simply wasn't charging enough for it. They will also tell you that it improves liquidity, but that doesn't make sense either. It's not like they're sitting on the shares keeping the markets moving. All it does is add a tax to trades.
IEX delays the transactions just enough that the nanosecond advantages that HFT enjoys can't be used to front run trades.
[+] [-] badosu|8 years ago|reply
[+] [-] tomalpha|8 years ago|reply
I'd guess in this instance that they'll skip both the TCP and IP layers completely and essentially be sending minimally encapsulated payloads out almost raw. Minimally encapsulated at this level might actually mean no encapsulation. Maybe some error-detection/CRC type stuff as these links tend to be super lossy.
[+] [-] 21|8 years ago|reply
[+] [-] JustSomeNobody|8 years ago|reply
SOH<header data>EOH STX<data>ETX CRC EOT
Then the receiver would send an ACK or NACK.
Repeat as necessary.
[+] [-] gaius|8 years ago|reply
[+] [-] gpderetta|8 years ago|reply
In fact the payload might be empty or mostly empty and data can be encoded in protocol fields (for example encoding symbol names in address fields as the link is otherwise point to point).
[+] [-] mhurd|8 years ago|reply
[+] [-] sniperinmahwah|8 years ago|reply