OMG this is a godsend! I've wanted to do webrtc outside of a browser in python for a while now but there was no available implementations (besides the native one, and I didn't want to go down the swig route).
Some questions:
Can it handle the signalling on its own?
Can it substitute any part of a webrtc session? That is, can I have either browser-aiortc, aiortc-browser or aiortc-aiortc?
Can it be used to build, say, a desktop application doing webrtc communication?
Signaling is a very app-specific question, as you can use a wide range of options. If you take a look at the "apprtc" example, it's a demo which uses appr.tc's signaling.
You can indeed use aiortc facing a browser, itself or any valid WebRTC endpoint.
Sure, you can use aiortc for a desktop application. However for this specific usecase embedding a browser which supports WebRTC (say QtWebEngine) might be a good option too.
Is there any specific area you spotted which would need some love? Modern spec compliance is most definitely a goal (e.g. RTCRtpTransceiver has been there from the start, aiortc uses the "modern" SDP form for data-channels), so any pointers are greatly appreciated!
I've had some great interactions with the Mozilla crowd (hey Lennart, hey Nils) - looking forward to more of the same!
This is super cool, great work! It's the second (mostly-)language-native implementation[1] outside of a browser that I'm aware of, both started this year despite the WebRTC spec being finalized about 5 years ago.
It's really important for libraries like this to exist so more people can build apps without binding to massive hard-to-compile browser codebases. Thank you for working on it!
Also kudos for going asyncio and Python 3 right from the gate.
--
[1] At least semi-functioning with DataChannel support. Lots of started attempts that have never gotten this far. The other implementation is in Go: https://github.com/pions/webrtc
Since a lot of WebRTC devs may be in this thread (and I personally didn't know of this project nor https://github.com/pions/webrtc) -- any erlang/elixir implementations out there?
I personally like the "server" and "apprtc" demos as you can easily talk to a browser. They illustrate slightly different things:
- "apprtc" relies on a third-party signaling service, and shows you how you can play media from a file / record it to a file, or generate video frame-by-frame.
- "server" shows you how you can combine media and signaling into a single Python-based server, and how you can apply image processing on the fly to the received video.
Great work! I can really appreciate the effort you likely put into re-implementing SCTP, since I'm currently working on implementing SCTP in Rust for the same reasons (WebRTC support). :)
That's awesome simmons! I can't wait to see Rust/Python/Go/Node all talking directly to each other. I really want to see a world where P2P communication is common.
It could really change how people design systems, have servers exchange data directly instead of overloading message brokers. Could have an even bigger impact on users! Instead we could see users exchanging files directly instead of depending on paid services!
I am the author of https://github.com/pions/webrtc we implemented SCTP, SRTP and working on DTLS now. So if you ever want to grab anything please do :) and always happy to chat if there is anything that could be better/any way we could help the rust implementation
That's good news, the WebRTC world is too much of a libusrsctp monoculture to my taste at the moment. Don't be surprised if you find some interesting edge cases in browsers as you stray from the beaten track :)
This is really impressive! Seems like a great way to learn WebRTC and potentially is going to be much more portable than the reference C++ implementation.
Isn’t C++ generally more portable than python? All you need is a sufficient compiler, as opposed to a correctly set up Python installation of the correct version.
Absolutely, might as well enjoy the rich Python ecosystem!
Starting with aiortc 0.9.9, there is now also deep integration with PyAV for all your FFmpeg needs.
- At the lowest level : aiortc uses PyAV's AudioFrame and VideoFrame class, and PyAV is soon to gain more ndarray converters: https://github.com/mikeboers/PyAV/pull/415
- At a higher level : aiortc provides MediaPlayer and MediaRecorder classes to read or write audio/video
Amazing stuff! Since I mainly do python, I'll definitely dig into this over the weekend.
Can this library in the current state record audio of both local and remote streams at the same time and could you mute a user for every participant on the call?
So far the main CPU hog is (unsurprisingly) media encoding, especially video. Luckily this occurs off the main thread, inside C function calls. Unless I'm mistaken the GIL is released, so there is no Python-specific downside on this specific point.
I've found heavy usage of asyncio.Queue to bridge consumers / producers does not lead to amazing performance, so I'm trying to cut down on that pattern in favour of.. callbacks (I know - not very asyncio-ish). A good testcase for testing performance without media encoding is transferring large amounts of data over datachannels (see datachannel-filexfer example). I usually hit ~ 70Mbps on the loopback interface, which is more than I've managed to get out of browsers.
I can't say I have experience with using aiortc at scale yet, so there are almost certainly so hotspots I haven't seen.
ORTC is what Microsoft has chosen to implement in Edge, so if you use WebRTC in Edge, you're most likely using the WebRTC-over-ORTC adaptor.
In short ORTC is: break WebRTC into objects you manipulate directly, instead of everything being instanciated via RTCPeerConnection. There is significant overlap between the two specs, for instance RTCRtpReceiver / RTCRtpSender.
[+] [-] znpy|7 years ago|reply
Some questions:
Can it handle the signalling on its own?
Can it substitute any part of a webrtc session? That is, can I have either browser-aiortc, aiortc-browser or aiortc-aiortc?
Can it be used to build, say, a desktop application doing webrtc communication?
[+] [-] jlaine|7 years ago|reply
You can indeed use aiortc facing a browser, itself or any valid WebRTC endpoint.
Sure, you can use aiortc for a desktop application. However for this specific usecase embedding a browser which supports WebRTC (say QtWebEngine) might be a good option too.
[+] [-] tw1010|7 years ago|reply
[+] [-] Orphis|7 years ago|reply
But I can only notice quite a bit of mismatch between your APIs and the current spec. Do you plan on updating your implementation to match it?
[+] [-] jlaine|7 years ago|reply
I've had some great interactions with the Mozilla crowd (hey Lennart, hey Nils) - looking forward to more of the same!
[+] [-] shazow|7 years ago|reply
It's really important for libraries like this to exist so more people can build apps without binding to massive hard-to-compile browser codebases. Thank you for working on it!
Also kudos for going asyncio and Python 3 right from the gate.
--
[1] At least semi-functioning with DataChannel support. Lots of started attempts that have never gotten this far. The other implementation is in Go: https://github.com/pions/webrtc
[+] [-] gfodor|7 years ago|reply
[+] [-] evadne|7 years ago|reply
Also somebody has forked erlyvideo to deal with WebRTC
[+] [-] ndesaulniers|7 years ago|reply
[+] [-] pranaysharma|7 years ago|reply
[+] [-] jlaine|7 years ago|reply
https://github.com/jlaine/aiortc/tree/master/examples
I personally like the "server" and "apprtc" demos as you can easily talk to a browser. They illustrate slightly different things:
- "apprtc" relies on a third-party signaling service, and shows you how you can play media from a file / record it to a file, or generate video frame-by-frame.
- "server" shows you how you can combine media and signaling into a single Python-based server, and how you can apply image processing on the fly to the received video.
[+] [-] simmons|7 years ago|reply
[+] [-] Sean-Der|7 years ago|reply
It could really change how people design systems, have servers exchange data directly instead of overloading message brokers. Could have an even bigger impact on users! Instead we could see users exchanging files directly instead of depending on paid services!
I am the author of https://github.com/pions/webrtc we implemented SCTP, SRTP and working on DTLS now. So if you ever want to grab anything please do :) and always happy to chat if there is anything that could be better/any way we could help the rust implementation
[+] [-] jlaine|7 years ago|reply
https://bugzilla.mozilla.org/show_bug.cgi?id=1443032
[+] [-] jswrenn|7 years ago|reply
Is your work on github/gitlab?
[+] [-] gfodor|7 years ago|reply
[+] [-] stochastic_monk|7 years ago|reply
[+] [-] ArtWomb|7 years ago|reply
[+] [-] jlaine|7 years ago|reply
Starting with aiortc 0.9.9, there is now also deep integration with PyAV for all your FFmpeg needs.
- At the lowest level : aiortc uses PyAV's AudioFrame and VideoFrame class, and PyAV is soon to gain more ndarray converters: https://github.com/mikeboers/PyAV/pull/415
- At a higher level : aiortc provides MediaPlayer and MediaRecorder classes to read or write audio/video
[+] [-] bkovacev|7 years ago|reply
Can this library in the current state record audio of both local and remote streams at the same time and could you mute a user for every participant on the call?
[+] [-] Dowwie|7 years ago|reply
[+] [-] jlaine|7 years ago|reply
I've found heavy usage of asyncio.Queue to bridge consumers / producers does not lead to amazing performance, so I'm trying to cut down on that pattern in favour of.. callbacks (I know - not very asyncio-ish). A good testcase for testing performance without media encoding is transferring large amounts of data over datachannels (see datachannel-filexfer example). I usually hit ~ 70Mbps on the loopback interface, which is more than I've managed to get out of browsers.
I can't say I have experience with using aiortc at scale yet, so there are almost certainly so hotspots I haven't seen.
[+] [-] rjsw|7 years ago|reply
[+] [-] detaro|7 years ago|reply
[+] [-] comesee|7 years ago|reply
[+] [-] londt8|7 years ago|reply
[+] [-] jlaine|7 years ago|reply
In short ORTC is: break WebRTC into objects you manipulate directly, instead of everything being instanciated via RTCPeerConnection. There is significant overlap between the two specs, for instance RTCRtpReceiver / RTCRtpSender.
[+] [-] jgh|7 years ago|reply