Interesting post! The author mentions that the first step was to "receive this input and to generate HLS output to a known folder". I have a couple questions about this step:
1: Did they re-encode the input? Or just repackage it?
2: I'm unfamiliar with EvoStream, but if it is ingesting RTMP and outputting HLS, why did Globo need to bother generating an HLS manifest? Couldn't they just use the one EvoStream created?
I am glad you liked. I am not the author of the blog post, but I was a member of the team.
1: We just repackage it.
2: The problem is that EvoStream stores the manifests and chunks locally and we needed high availability. That's why we use an external data storage. We have had up to 30 simultaneous streams, with 7 bitrates each and 2 hours of DVR.
Honestly - one of the most informative, well written, extremely well referenced articles I've read literally in years.
This article will teach me more than I've learned in a long time. Thank you very much for your contributions..
It is rewarding working at globo.com with such skilled engineers and great solutions, in almost every area. Congrats to Leandro, Juarez, and everybody involved.
It feels like they reinvented the wheel on a lot of things (especially on the monitoring side). I also wish there was some more detail about the caching/endpoint side of things. We rely heavily on Akamai's services for their edge servers, caching, and site acceleration. I don't see anything like that here, but maybe they're just not mentioning it?
We only implemented the dashboard (getting data from all kinds of source) on monitoring side, we relied on: logtash, graphite, elasticsearch...
The architecture up from what was described is almost purely to provide caching capabilities, aka bunch of edge servers caching content to final users.
Please keep in mind that FIFA14 was one event, we still broadcast many other events and channels simultaneously ( private and public).
About why chose HLS over RTMP, in our experiments HLS showed to be easier to scale than RTMP (maybe because the latter is stateful + the current players don't have an optimized adaptive bitrate algorithm + it's easier to scale http over rtmp)
Working in the TV industry myself, the uninevitable question on DRM springs to mind. Was these streams encrypted as well somewhere along the described process?
This documentary is very old and pre-dates the internet.
There's no denying the extent of Globo's influence. But, compared to the 80's, it's merely a shadow of it's former self. Probably because of the internet. Also, there are other communication vehicles fighting for eyeballs now.
Politics aside, its IT arm 'globo.com' is pretty modern and open.
[+] [-] jsaxton86|11 years ago|reply
1: Did they re-encode the input? Or just repackage it?
2: I'm unfamiliar with EvoStream, but if it is ingesting RTMP and outputting HLS, why did Globo need to bother generating an HLS manifest? Couldn't they just use the one EvoStream created?
[+] [-] jbochi|11 years ago|reply
1: We just repackage it. 2: The problem is that EvoStream stores the manifests and chunks locally and we needed high availability. That's why we use an external data storage. We have had up to 30 simultaneous streams, with 7 bitrates each and 2 hours of DVR.
[+] [-] kweks|11 years ago|reply
[+] [-] emilsedgh|11 years ago|reply
What is the benefit of EvoStream over nginx-rtmp? I use nginx-rtmp as well and it creates HLS files just fine.
Also, how did you handle the network load? How many edge servers were actually delivering the content to users?
Did you use 10G ethernets, link bonding, or did you use cloud services (eg cloudfront) to deliver the content?
Thanks a lot.
[+] [-] dreampeppers99|11 years ago|reply
* We used 20G ethernet, we could get 19G from each machine.
* All the content is hosted by us.
[+] [-] gregor7777|11 years ago|reply
[+] [-] icaromedeiros|11 years ago|reply
[+] [-] alexkappa|11 years ago|reply
[+] [-] dreampeppers99|11 years ago|reply
You can send metrics from Cassandra to graphite http://www.datastax.com/dev/blog/pluggable-metrics-reporting...
Since the streaming is similar to HTTP page flow, it's not that hard.
Ex: http://blazemeter.com/blog/how-load-test-http-live-media-str...
[+] [-] Finster|11 years ago|reply
[+] [-] dreampeppers99|11 years ago|reply
The architecture up from what was described is almost purely to provide caching capabilities, aka bunch of edge servers caching content to final users.
[+] [-] robinhowlett|11 years ago|reply
Why not just use Akamai for RTMP ingest and delivery? From my experience they can do all that was described and at scale.
[+] [-] dreampeppers99|11 years ago|reply
About why chose HLS over RTMP, in our experiments HLS showed to be easier to scale than RTMP (maybe because the latter is stateful + the current players don't have an optimized adaptive bitrate algorithm + it's easier to scale http over rtmp)
[+] [-] xai3luGi|11 years ago|reply
[+] [-] superted|11 years ago|reply
[+] [-] eder_roger|11 years ago|reply
[+] [-] wjsf|11 years ago|reply
[+] [-] robertomb|11 years ago|reply
[+] [-] dreampeppers99|11 years ago|reply
[+] [-] flavioribeiro|11 years ago|reply
[+] [-] aprdm|11 years ago|reply
[+] [-] rarararararara|11 years ago|reply
This documentary is still banned in Brazil and can't be broadcast offline anywhere.
[+] [-] outworlder|11 years ago|reply
There's no denying the extent of Globo's influence. But, compared to the 80's, it's merely a shadow of it's former self. Probably because of the internet. Also, there are other communication vehicles fighting for eyeballs now.
Politics aside, its IT arm 'globo.com' is pretty modern and open.