Periscope developed a Low-Latency HTTP Live Streaming (LHLS) technique that relies on HTTP chunked transfer-encoding to stream video bytes as they are encoded at the origin. This is still subject to TCP packet retransmission overhead, but the time-to-first-byte is reduced significantly and leads to less buffering on the client.
Most systems that serve HLS media use fixed content-length segments, which requires knowledge of the length of a segment before the first byte can be sent over the wire. So, for a 5 second segment you would need to encode the entire 5 seconds before the first byte can be sent; this does not apply when streaming the segments with chunked transfer encoding.
Incidentally, at Mux we also use chunked transfer-encoding to stream video that is encoded on-the-fly with great performance.
I've heard from colleagues that this won't be possible with DASH due to the switch to fMP4 format. One of my co-workers tells me that fMP4 requires the entire segment to be loaded before playback can begin while TS segments don't require this. We've been looking into very small segments (e.g. 1s duration) to reduce latency but I've been interested in the LHLS approach since I first heard of it.
Low delay is much, much more important for calls than for streaming. One second of buffering delay may be acceptable in streaming playback (users often contend with longer delays). That much delay will severely degrade a video call, especially if the audio stays synced with the delayed video.
That really depends on the service. For on-demand services like Netflix or YouTube your argument is a fair one. But for live services it matters a lot more - particularly in broadcasting. Even more so if your clients are watching the same feed and using that to determine when to inject ads (eg in a live sporting event). Then it quickly becomes very important to keep latency as low as possible.
What is the distinction that you're making here? A video call seems like a use-case of video/audio streaming, not something separate. There are cases where low-latency streaming isn't important, but the article is talking about cases where it is.
Anybody have an idea how to put this into practice? I recently tried streaming video over wifi from a Raspberry (for a robotics hobby project) , and everything I tried was either unusable or very delayed.
Is there an open source low latency video streaming solution for hobbyists?
Anyone know what the best streaming solution for browser <-> browser video calling is? It probably has to be built on top of WebRTC but I'm wondering if there are codecs and forward error correction algorithms out there already in Javascript to use.
For small numbers of people in a call (say, n < 5), the "best" -- meaning lowest latency -- solution is direct browser to browser WebRTC connections. Both Chrome's and Firefox's WebRTC implementations have quite good FEC built in. And sending UDP packets directly between peers will have much lower latency than routing through a media server.
Of course, sometimes peer-to-peer won't work for you. Maybe you have requirements that push you towards routing media through a server. (Content filtering, or compositing video or mixing audio, for example.) Or maybe you have more than a few people in a call. If so, upstream bandwidth and encoding become bottlenecks for mesh/peer-to-peer. Finally, some firewalls won't allow UDP traffic from/to computers behind them, so you'll need to route UDP through a central server, or (much worse) tunnel over TCP.
Back on the subject of latency and error correction in WebRTC, here are some fun links:
Mozilla article from when they first turned on Opus FEC. Includes sample audio for calls with 19% packet loss. (19% packet loss is very, very bad. My startup makes a browser-to-browser video calling tool, and we try hard to deal well with packet loss that high, but it's a losing battle.) https://blog.mozilla.org/webrtc/audio-fec-experiments/
The title is misleading, when they say low-delay, really they mean over UDP instead of TCP. Their answer to video over UDP is a marginal efficiency gain in a forward error correction algorithm called RLNC.
I haven't looked at RLNC, but it does seem to have some gains over more traditional FEC schemes.
As stated in the article, low delay is very hard to achieve over a reliable transport based on re-transmissions (such as TCP) - you have to wait for the loss to be detected and then the retransmitted packet to arrive. This is at least 3 x the latency of the link. Therefore if you really care about bounded low latency you need to use some form of erasure correcting algorithm (to provide upfront redundancy).
In the article we simply show that substituting one "old" algorithm for a more modern one can give you much better efficiency (protection against packet loss) for the same bandwidth and latency/delay budget.
Anyone know what PSNow/ cloud gaming services use? I’m using it to play some ps3 games and the delay is immediately noticeable when you start playing, but your brain adjusts and it’s not noticeable. Has to be around 300-400 ms round trip
[+] [-] urlgrey|8 years ago|reply
Here's a Periscope post about LHLS: https://medium.com/@periscopecode/introducing-lhls-media-str...
Most systems that serve HLS media use fixed content-length segments, which requires knowledge of the length of a segment before the first byte can be sent over the wire. So, for a 5 second segment you would need to encode the entire 5 seconds before the first byte can be sent; this does not apply when streaming the segments with chunked transfer encoding.
Incidentally, at Mux we also use chunked transfer-encoding to stream video that is encoded on-the-fly with great performance.
[+] [-] reggieband|8 years ago|reply
[+] [-] kazinator|8 years ago|reply
[+] [-] laumars|8 years ago|reply
[+] [-] khedoros1|8 years ago|reply
[+] [-] tzahola|8 years ago|reply
[+] [-] MiniMax42|8 years ago|reply
[+] [-] fenesiistvan|8 years ago|reply
[+] [-] nh2|8 years ago|reply
[+] [-] jakobegger|8 years ago|reply
Is there an open source low latency video streaming solution for hobbyists?
[+] [-] mbrumlow|8 years ago|reply
All the code is on GitHub. I can get very low latency video. Most of the delay comes in the form of the speed of light being so slow.
On the web site in use jsmpeg.
[+] [-] Jakob|8 years ago|reply
[+] [-] imtringued|8 years ago|reply
Try using kurento [0].
[0] https://www.kurento.org/
[+] [-] mgamache|8 years ago|reply
[+] [-] coldsauce|8 years ago|reply
[+] [-] kwindla|8 years ago|reply
Of course, sometimes peer-to-peer won't work for you. Maybe you have requirements that push you towards routing media through a server. (Content filtering, or compositing video or mixing audio, for example.) Or maybe you have more than a few people in a call. If so, upstream bandwidth and encoding become bottlenecks for mesh/peer-to-peer. Finally, some firewalls won't allow UDP traffic from/to computers behind them, so you'll need to route UDP through a central server, or (much worse) tunnel over TCP.
Back on the subject of latency and error correction in WebRTC, here are some fun links:
Draft spec for FEC in WebRTC: https://tools.ietf.org/html/draft-ietf-rtcweb-fec-08
Mozilla article from when they first turned on Opus FEC. Includes sample audio for calls with 19% packet loss. (19% packet loss is very, very bad. My startup makes a browser-to-browser video calling tool, and we try hard to deal well with packet loss that high, but it's a losing battle.) https://blog.mozilla.org/webrtc/audio-fec-experiments/
Notes from the very knowledgeable folks at Callstats.io about WebRTC FEC. Covers some of the same material as this thread's original post: https://www.callstats.io/2016/11/09/how-to-recover-lost-medi...
Tsahi Levent-Levi's benchmarks showing how a few different media servers perform in the context of 10% packet loss: https://testrtc.com/webrtc-media-server-packet-loss/
[+] [-] stanlarroque|8 years ago|reply
[+] [-] thefourthchime|8 years ago|reply
I haven't looked at RLNC, but it does seem to have some gains over more traditional FEC schemes.
https://arxiv.org/abs/1802.04873
[+] [-] mortenvp|8 years ago|reply
In the article we simply show that substituting one "old" algorithm for a more modern one can give you much better efficiency (protection against packet loss) for the same bandwidth and latency/delay budget.
[+] [-] rjeli|8 years ago|reply