If you ever were in the situation to try to find out why the video quality of your WebRTC call was not good, you probably have also sworn at the encrypted RTP and RTCP. Instead of trying to put log statements into your locally compiled Firefox version, you can now simply request logging of the RTP and RTCP packets.
This isn’t a post about a new WebRTC feature. Rather, we’ll explore the new
Let us talk about Firefox’s backend audio system. Every sound you hear while you are watching a video or using a WebRTC service comes from the backend audio library. The role of the library is to communicate with audio devices and to provide audio input and/or output. This library is called ‘cubeb’ and it is part of the Firefox source code.
You can view a recording of the Q&A session on WebRTC standards from Monday 2/27/17 here.
More of last year’s new laptop cameras have few or no low-resolution modes. These reveal a big difference in how Chrome and Firefox implement the
getUserMedia camera API. Whereas Chrome re-scales video to whatever constraints you put in, Firefox does not, instead guaranteeing the closest native resolution. Both are technically within spec. That said, the elaborate constraints language offered by
max ranges and
ideal values, was clearly designed for resolution discovery; it is overkill for setting a target resolution override.
But sending large resolution video over a peer connection is expensive. Here’s how you downscale it in Firefox (more…)
Firefox has supported
getStats to the spec for a couple of years now, and Chrome Canary finally has a spec-compliant version, so it’s a good time to check in how to write code that (soon!) will work across browsers.
Testing WebRTC is challenging. It is inherently timeout based. Tests that work well locally often fail when run in automation. At Mozilla, we run the majority of our Android tests on ARM emulators on AWS instances. In case anyone is wondering, this is very slow. We’ve recently started running our tests on actual phones as well as on emulators. Running faster should lead to fewer performance related intermittent failures.
Screen-sharing is a powerful new web feature that lets you share what’s on your computer screen with a web site. This can be extremely useful. Imagine co-browsing with a friend, or having a service technician remotely diagnose a problem on your computer. But at the same time, it carries significant security and privacy risks.
Certain windows are riskier to share than others. Firefox will warn you not to share browser windows, or even your entire screen when a browser window is present on it, unless you trust the web site. The reasons for this are technical, but boil down to this: (more…)
A really enhanced webcam stream!
Recently, HTMLCanvasElement.captureStream() was implemented in browsers. This allows you to expose the contents of a HTML5 canvas as a MediaStream to be consumed by applications. This is the same base MediaStream type that getUserMedia returns, which is what websites use to get access to your webcam.
The first question that comes to mind is, of course: “Is it possible to intercept calls to getUserMedia, get a hold of the webcam MediaStream, enhance it by rendering it into a canvas and doing some post-processing, then transparently returning the canvas’ MediaStream?”
As it turns out, the answer is (more…)
In Firefox 49 we released support for ICE Consent Freshness as a first way to detect that an ICE transport is no longer working. What does ICE Consent Refresh do, you ask?
With plain ICE
Lets start with what we had before. Without Consent Refresh the ICE standard requires us to send so called Keepalives to keep the ports in the NAT open. But these Keepalives had a couple of issues. First of all the Keepalives only send traffic. They do not require a response nor do they expect any response. So as the sender, you basically try your best to keep the NAT ports open, but you have no idea if it works or not.
Adding Consent timeouts
To improve that experience, in Firefox 49 we released support for ICE Consent Freshness. With plain ICE you stop sending STUN binding requests once both parties have identified and agreed on a working transport path. The idea behind ICE Consent Freshness is that once connected you continue to send STUN binding requests. And the other side needs to answer.
So every 5 seconds Firefox (version >= 49) sends another binding request no matter if the ICE transport is in use or not, and it expects the other side to reply with a binding response. If it hasn’t received a binding response for 6 consecutive binding requests, in other words no reply within the last 30 seconds, it will give up and mark the transport as failed. This results in switching the ICE connection state to ‘failed‘ and stop sending any packets over that transport.
This also protects the network from unnecessary traffic as Firefox before would have continued to send RTP (assuming the application logic had not closed the PeerConnection by now). By the way, as a receiver of these Consent Freshness binding requests, you can also revoke the consent by responding with a 403 binding response error.
Now when your WebRTC application watches the ICE connection state it at least knows when the transport layers have given up. However, 30 seconds is quite a long time for an inpatient human who was just enjoying a thrilling conversation. And it is not advisable to try an ICE restart at this point.
Utilizing Consent to detect connection problems
The new feature, which utilizes the previous ICE Consent Freshness work, is that Firefox (version >= 52) switches the ICE connection state even earlier. Basically, when not receiving a reply for one of the binding requests within 5 seconds, the ICE connection state is going to switch to ‘disconnected‘. At this point the senders can still send packets. And more binding requests are going to be sent. If one of the subsequent binding requests gets a binding response, the ICE connection state switches back to ‘connected‘. And if no binding responses are received at all, approximately 25 seconds later the ICE connection state switches to ‘failed‘ and no further packets are sent.
So it’s up to the application to decide what to do when the ICE connection state switches from ‘connected‘ to ‘disconnected‘. One option is to try an ICE restart right then. But since the stun binding requests for ICE Consent Freshness are not being retransmitted, it could also mean that just the binding response packet got dropped by the network. The better alternative is to wait for a couple of more seconds to see whether the ICE connection state switches back to ‘connected‘, and if not, try the ICE restart prior to the connection state changing to ‘failed‘.
Picture By Brocken Inaglory (Own work) [GFDL (http://www.gnu.org/copyleft/fdl.html), CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/) or CC BY-SA 2.5-2.0-1.0 (http://creativecommons.org/licenses/by-sa/2.5-2.0-1.0)], via Wikimedia Commons