I have been working on a large WebRTC project, and it’s a great technology but it’s just so fragmented right now it’s hard.
For iOS
1. Safari has a high pass filter or something on the incoming audio. Send in some music on opus/48000 with the proper maxbitrate and listen. Now download chrome for iOS and listen. Chrome has MUCH more low end. Running the audio before and after into a spectrum analyzer chrome is much closer to the original sound.
2. If you make the video full screen then exit full screen if pauses.
3. I couldn’t get incoming RTP that is opus/48000/2 to be played in stereo. But as number 1 playing it in chrome does play in stereo.
Android isn’t much better.
1.The fragmentation is such a HUGE pain. The manufacturer browsers on the phones seem to be very hit or miss. How many browsers come by default on android phones now days?
2. The latest Samsung phones cameras can’t do resolution 640x360 for whatever reason. Previous generations could and other android phones can as well.
Even desktop isn’t all there.
1. If you want to webrtc screen share there is no way to automatically select a screen or tab, not even CLI flags. So you can’t automate testing it. And you can’t even click the dialog in the chrome remote debugger.
2. FireFox has nothing like chromes chrome://webrtc-internals
A few other things
1. Simulcasting is ghetto
2. I wish server CPUs would support h264/vp8/vp9 hardware encoding and decoding on the CPUs
3. Want to collect metrics with getStats? Well there is no easy way to record all of them and the one SaaS is super super expensive
This is one of the reasons we’ve had to go the dreaded electron route and really push the desktop app: it’s just not feasible to deliver the same UX across browsers/OS. Really frustrating. I’ll add one to your list:
4. I wish RTCRtpTransceiver would allow reusing encoded MediaStreams instead of re-encoding. For video, dropped frames are acceptable without needing to renegotiate bandwidth.
We've been watching Apple undermine WebRTC for years now. There's been nothing stopping them from taking the existing library and running with it; nothing stopped them for the years they had no support, and nothing is stopping them now that they have nominal support.
Manufacturer browsers used to be a big deal, but for the last couple years it's been mostly Chrome or something nearly identical on most devices. Not having that specific resolution available is one thing, but it's nothing like having no support, or inexplicably degrading audio quality.
> If you want to webrtc screen share there is no way to automatically select a screen or tab, not even CLI flags. So you can’t automate testing it. And you can’t even click the dialog in the chrome remote debugger.
I've recorded a tab automatically before with Chrome [0]. Basically I have --auto-select-desktop-capture-source set to pick a tab then named my tab something it could find, you can probably get close with entire screen too.
> Simulcasting is ghetto
Agreed, but in my recent server side project I just realized if I ignore the ssrcs and use the rids from the rtp packets, my sfu did what I want. I found while working with Chrome, upping the log visibility really helped me understand why, based on estimated bitrate sent by the server, my highest simulcast stream wasn't being sent (since then I adhere to remb needs).
> I wish server CPUs would support h264/vp8/vp9 hardware encoding and decoding on the CPUs
You should look into SVC. Nowadays there aren’t too many reasons why you’d need to decode/reencode if all you want is to change resolution, quality or drop frames graciously.
It is not in Google or Apple's best interest to work together on things like this, because then it'd accelerate the already fast-coming age of Progressive Web Apps (PWAs), which removes their, what, 30% cut from "apps"?
Which are all....websites smashed into "native" code + privacy invading trackers, anyway?
quick question - how do you scale webrtc server-side ?
Do you use kubernetes to scale jitsi/mediasoup/whatever servers up and down ?
what kind of loadbalancers do you use, etc . I come from the kubernetes side and have never read anything about scaling of these things...so am super curious.
I was pretty unimpressed by Zoom as I used to think along the lines of "bro just open up a WebRTC connection. You can use socket.io it's super simple"
Then... I attended a "Zoom Nightclub" and saw over 200 attendees streaming video to one another. All encryption, routed through China, etc. issues aside; I was very very impressed.
Another nitpick: WebRTC connections are extremely fragile and a pain to herd. We’ve had to build both data and audio keepalives into Squawk[1] to ensure that we can hold connections for long durations (running in the background 24/7). And if the connection encounters a problem, often the only recourse is to renegotiate or recycle.
EDIT: For anyone curious, we swap out the track from the RTCRtpSender so we’re not transmitting any data when we’re muted (muting the track locally means you’re still sending frames of silence every 20ms which eats up cpu and bandwidth). And every 15 seconds we send a 60ms blast of silence, and verify on the receiver every minute.
Why not just use ICE connectivity checks? That's what they are for.
And I have proposed additions to the WebRTC API to allow control of when ICE checks are sent and how long a particular ICE candidate pair should stay alive (you could set it to "forever"):
I'm using webRTC for a video chat project that I'm developing now. I had a hard time making it work on iOS browsers and ended up supporting iOS Safari alone. Though all iOS browsers use the same WebKit, Apple still reserves a few things only for Safari I guess (maybe H264/VP8 video codec related stuff?!).
It may take some time for webRTC (or Apple) to get there, to webRTC become a solid option for p2p video communications.
If anyone wants to quickly try webRTC, check this demo https://appr.tc/ out, you know, with multiple browsers/tabs.
ps: I used "simple-peer" library and it's quite good for beginners.
WebRTC mildly competes with their proprietary FaceTime URLs that, along with iMessage, "encourages" people to keep coming back into their stores. They could make it work exactly like the implementations on Windows and Linux (and the other macOS web browsers that are "required" to use WebKit) if they desired. You can still use an SDK though --as long as they allow you publish it to users.
Author of Peer Calls here, an open source, anonymous WebRTC multi-user conferencing solution. I recently ported the app that Go and added an SFU (thanks to pion/webrtc) to make support for 3+ users better. Works on all recent major browsers (including Safari on iOS).
In the past there was no encryption for communications then sRTP with DTLS became the way of doing it (secure signalling and media) when you negotiate sRTP material between parties you can intercept the keys and inject some pretending to be the remote caller and viceversa. https://bloggeek.me/is-webrtc-safe/ a good read
A bit off topic: recently multi monitor screen sharing based on WebRTC was fixed in Chromium/Linux/x11 (landed in Chrome 83) making my Google Meet/ Jitsi screen sharing support sessions much easier...
Yesterday Vivaldi & Brave screensharing on Fedora were dismal with Whereby (which uses WebRTC assumingely) while Chrome 83 worked great. Why would that be? Because Whereby doesn’t test Chromium browsers on Linux?
Why is `RTCPeerConnection.setLocalDescription()` an independent method? I've always wondered what use case might exist that would not set the local description right after calling `pc.createOffer()` or `pc.createAnswer()`. Similarly with `pc.setRemoteDescription()`.
Wouldn't have been clearer to have createOffer + processAnswer (for a negotiation started locally), and processOffer + createAnswer (to respond to remote offers), and set the local/remote descriptions implicitly as part of these methods?
`setLocalDescription` is where all the logic happens! `createOffer`/`createAnswer` just creates an easy config to pass.
You can change lots of undocumented stuff in the SDP! This list isn't exhaustive, but I have seen production apps do these.
* Change ice ufrag/upwd
* Enable simulcast
* Change code preferences
* Change codec specific behavior (Opus ptime)
My guess is that it works this way so browser vendors don't have to actually standardize behavior. They don't need to go through the W3C every time they want add something new. ORTC was released to try and fix this though. `SDP Munging` is a terrible idea and has caused me so many headaches :/
I needed a very low traffic chat feature for a website built on Django so i went looking too what existing code there was.
1 was based on simple Ajax to server and a server backed message store.
About a dozen used WebRTC or other advanced techniques.
I wonder how many people who could quite easily cope with the expected traffic using brain-dead simple techniques end up going the complex route because they assume that's how you do chat?
Is your chat text only? I think for media WebRTC is going to always win in the browser (unless you have the developers who can implement custom congestion control etc..)
For text only I still see a couple advantages
* E2E Security. With WebRTC I don't have to worry about snooping. Why should I upload my messages to a server?
* Scaling/Bandwidth/Ops Burden. Massively reduced if all my clients are directly connected to each other
* Ajax (TCP) can't do as much. DataChannels give me lossy/out-of-order messaging. Not useful for chat, but really great if building anything gaming/real-time
Before that, a webserver was still required to initiate the connection between 2 user-peers, which of course made little sense. I once asked about it and got an answer about "DDOS risks", and it made sense, because if you let people visit some webpage, it would be an easy way to DDOS.
You need to provide more details. How does it compare to other chat applications? What improvements and features does it have that other chat applications don't have.
[+] [-] nerdbaggy|5 years ago|reply
For iOS
1. Safari has a high pass filter or something on the incoming audio. Send in some music on opus/48000 with the proper maxbitrate and listen. Now download chrome for iOS and listen. Chrome has MUCH more low end. Running the audio before and after into a spectrum analyzer chrome is much closer to the original sound.
2. If you make the video full screen then exit full screen if pauses.
3. I couldn’t get incoming RTP that is opus/48000/2 to be played in stereo. But as number 1 playing it in chrome does play in stereo.
Android isn’t much better.
1.The fragmentation is such a HUGE pain. The manufacturer browsers on the phones seem to be very hit or miss. How many browsers come by default on android phones now days?
2. The latest Samsung phones cameras can’t do resolution 640x360 for whatever reason. Previous generations could and other android phones can as well.
Even desktop isn’t all there.
1. If you want to webrtc screen share there is no way to automatically select a screen or tab, not even CLI flags. So you can’t automate testing it. And you can’t even click the dialog in the chrome remote debugger.
2. FireFox has nothing like chromes chrome://webrtc-internals
A few other things
1. Simulcasting is ghetto
2. I wish server CPUs would support h264/vp8/vp9 hardware encoding and decoding on the CPUs
3. Want to collect metrics with getStats? Well there is no easy way to record all of them and the one SaaS is super super expensive
[+] [-] zumachase|5 years ago|reply
4. I wish RTCRtpTransceiver would allow reusing encoded MediaStreams instead of re-encoding. For video, dropped frames are acceptable without needing to renegotiate bandwidth.
[+] [-] microcolonel|5 years ago|reply
Manufacturer browsers used to be a big deal, but for the last couple years it's been mostly Chrome or something nearly identical on most devices. Not having that specific resolution available is one thing, but it's nothing like having no support, or inexplicably degrading audio quality.
[+] [-] kodablah|5 years ago|reply
I've recorded a tab automatically before with Chrome [0]. Basically I have --auto-select-desktop-capture-source set to pick a tab then named my tab something it could find, you can probably get close with entire screen too.
> Simulcasting is ghetto
Agreed, but in my recent server side project I just realized if I ignore the ssrcs and use the rids from the rtp packets, my sfu did what I want. I found while working with Chrome, upping the log visibility really helped me understand why, based on estimated bitrate sent by the server, my highest simulcast stream wasn't being sent (since then I adhere to remb needs).
0 - https://github.com/cretz/chrome-screen-rec-poc/blob/master/a...
[+] [-] jupp0r|5 years ago|reply
You should look into SVC. Nowadays there aren’t too many reasons why you’d need to decode/reencode if all you want is to change resolution, quality or drop frames graciously.
[+] [-] ptman|5 years ago|reply
[+] [-] ta17711771|5 years ago|reply
Which are all....websites smashed into "native" code + privacy invading trackers, anyway?
[+] [-] sandGorgon|5 years ago|reply
Do you use kubernetes to scale jitsi/mediasoup/whatever servers up and down ?
what kind of loadbalancers do you use, etc . I come from the kubernetes side and have never read anything about scaling of these things...so am super curious.
[+] [-] barbegal|5 years ago|reply
Why is this? What advantages are there to doing it on a CPU rather than in a GPU?
[+] [-] ccmcarey|5 years ago|reply
[+] [-] SeeDave|5 years ago|reply
Then... I attended a "Zoom Nightclub" and saw over 200 attendees streaming video to one another. All encryption, routed through China, etc. issues aside; I was very very impressed.
[+] [-] dudus|5 years ago|reply
[+] [-] TACIXAT|5 years ago|reply
[+] [-] paxys|5 years ago|reply
[+] [-] zumachase|5 years ago|reply
[1] https://www.squawk.to
EDIT: For anyone curious, we swap out the track from the RTCRtpSender so we’re not transmitting any data when we’re muted (muting the track locally means you’re still sending frames of silence every 20ms which eats up cpu and bandwidth). And every 15 seconds we send a 60ms blast of silence, and verify on the receiver every minute.
[+] [-] pthatcherg|5 years ago|reply
And I have proposed additions to the WebRTC API to allow control of when ICE checks are sent and how long a particular ICE candidate pair should stay alive (you could set it to "forever"):
https://github.com/w3c/webrtc-ice/pull/22
[+] [-] memexy|5 years ago|reply
[+] [-] gopikrishnan|5 years ago|reply
It may take some time for webRTC (or Apple) to get there, to webRTC become a solid option for p2p video communications.
If anyone wants to quickly try webRTC, check this demo https://appr.tc/ out, you know, with multiple browsers/tabs.
ps: I used "simple-peer" library and it's quite good for beginners.
[+] [-] maxwell|5 years ago|reply
https://caniuse.com/#feat=stream
The Judiciary Committee asked Apple about this last summer, see question 6:
https://docs.house.gov/meetings/JU/JU05/20190716/109793/HHRG...
I think it's close to landing though, need to revisit.
[+] [-] vaxman|5 years ago|reply
[+] [-] sdan|5 years ago|reply
[+] [-] Sean-Der|5 years ago|reply
* Open Source classic game streaming - https://github.com/giongto35/cloud-game
* Controlling industrial construction equipment - https://twitter.com/shirokunet/status/1268011883816054784
* Run a web browser remotely, great for low spec computers - https://github.com/nurdism/neko
* Use NAT Traversal to access hosts. No more bastion hosts/3rd party services - https://github.com/mxseba/rtc-ssh
* Use Tor via WebRTC DataChannels - https://snowflake.torproject.org/
* Fly a drone via WASD in the browser - https://github.com/oliverpool/tello-webrtc-fpv
* Share files P2P in a cross-platform way. I HATE paying to transfer files, seems so basic - https://github.com/saljam/webwormhole
* Access a friends terminal from your browser via P2P - https://github.com/maxmcd/webtty
[+] [-] jeremija|5 years ago|reply
See https://github.com/peer-calls/peer-calls for more information.
You can host it yourself or use peercalls.com
Happy to answer any questions in this thread.
[+] [-] bitminer|5 years ago|reply
[+] [-] spicyramen|5 years ago|reply
[+] [-] tkubacki|5 years ago|reply
[+] [-] smartbit|5 years ago|reply
[+] [-] j1elo|5 years ago|reply
Wouldn't have been clearer to have createOffer + processAnswer (for a negotiation started locally), and processOffer + createAnswer (to respond to remote offers), and set the local/remote descriptions implicitly as part of these methods?
[+] [-] Sean-Der|5 years ago|reply
You can change lots of undocumented stuff in the SDP! This list isn't exhaustive, but I have seen production apps do these.
* Change ice ufrag/upwd
* Enable simulcast
* Change code preferences
* Change codec specific behavior (Opus ptime)
My guess is that it works this way so browser vendors don't have to actually standardize behavior. They don't need to go through the W3C every time they want add something new. ORTC was released to try and fix this though. `SDP Munging` is a terrible idea and has caused me so many headaches :/
[+] [-] gopikrishnan|5 years ago|reply
[+] [-] andybak|5 years ago|reply
1 was based on simple Ajax to server and a server backed message store.
About a dozen used WebRTC or other advanced techniques.
I wonder how many people who could quite easily cope with the expected traffic using brain-dead simple techniques end up going the complex route because they assume that's how you do chat?
[+] [-] Sean-Der|5 years ago|reply
For text only I still see a couple advantages
* E2E Security. With WebRTC I don't have to worry about snooping. Why should I upload my messages to a server?
* Scaling/Bandwidth/Ops Burden. Massively reduced if all my clients are directly connected to each other
* Ajax (TCP) can't do as much. DataChannels give me lossy/out-of-order messaging. Not useful for chat, but really great if building anything gaming/real-time
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] jokoon|5 years ago|reply
[+] [-] ashok_tankala|5 years ago|reply
I built a simple messenger using WebRTC. You can check at http://sambhashana.com/.
Please use it and share your feedback
[+] [-] memexy|5 years ago|reply
[+] [-] panpanna|5 years ago|reply
How hard is it to add real-time voice communication to a web page using this?
Can it be done without using third party libraries and frameworks?
[+] [-] gopikrishnan|5 years ago|reply
https://webrtc.github.io/samples/src/content/peerconnection/...
code: https://github.com/webrtc/samples/tree/gh-pages/src/content/...