February 21, 2024

End-to-end-encrypt WebRTC in all browsers!

Contributed by Jan-Ivar Bruaroey,

Hi all, we haven’t blogged since the WebRTC API was standardized back in January of 2021! But now in 2024 folks are asking us what’s up with all the new APIs? — We’re coming off an exciting second period of innovation and experimentation in WebRTC, so we’re brushing off the old blogging muscles to cover what’s new, in what’s promising to be a new series of posts.

First, don’t worry. WebRTC 1.0 has been tremendously successful and has not fundamentally changed. Instead, its success has led to developers wanting to use it for a variety of things. This demand has driven continued collaboration in the W3C  to standardize new functionality on top of the 1.0 baseline. Each new API exposes exciting new functionality, and each one deserves its own blog post, so let’s start right away with one of them:

Today’s topic is End-to-End-Encryption (E2EE) in WebRTC.

First, we should note that WebRTC is already peer-to-peer encrypted by DTLS, so there’s no need to use additional APIs except to protect data against any middle-boxes your service might employ as WebRTC end-points. That said…

Did you know all major browsers have an API to encrypt WebRTC calls end-to-end? They do! The caveat is the API shape differs slightly between browsers right now. But don’t despair! We’re here to cover that gap. If you’ve been with us for a while you know we’ve been here before, and this is how the sausage is made.

A worker-first API

Chromium experimented early with shipping APIs for this. Unfortunately the early APIs exposed the media pipeline on main-thread, subjecting it to risks of jank given the nature of the JavaScript event model. The Working Group learned from these experiments, and settled instead on a “worker-first” API, which means an API that is simpler to use in a worker than from main-thread.

This standards-track API is RTCRtpScriptTransform, and lets you transform encoded frames before sending, and back again on reception. The most obvious use-case is E2EE, but it can also be used for other things like adding metadata.

See RTCRtpScriptTransform in action below, XOR-ing data on send and receive (runs in all browsers):

  1. Click Start! and share your camera, and you’ll see two videos: what’s sent and what’s received by a peer
  2. Uncheck ✅ descramble to skip XOR on receive for some spectacle (the garbage illustrates that the receiver was “decrypting” with XOR before!)

Replace XOR in a real application with crypto of course, using keys secret to your application. Disclaimer: This is not zero-knowledge E2EE since you hold the keys — Mozilla wanted the stronger SFrameTransform API, but that’s in limbo — so until something stronger is standardized, this is what’s available, and protects end-users mostly against middle-boxes, not the content service provider (you).

How it works

Click on the “JavaScript” tab above to see the example code. Sender-side we add a transform like this:
sender.transform = new RTCRtpScriptTransform(worker, {side: "send"});

…and receiver-side it looks the same, since it’s XOR-ing (the side parameter is for the ✅ checkbox):
receiver.transform = new RTCRtpScriptTransform(worker, {side: "receive"});

And that’s (almost) it! Scroll down to see the worker in action. On the worker-side, things get exposed here:
onrtctransform = async ({transformer: {readable, writable, options}}) => {

…and the worker connects the redable to the writable through a custom (XOR) TransformStream:
await readable.pipeThrough(new TransformStream({transform})).pipeTo(writable);

And that’s it!

The rest is standard worker stuff. A "descramble" message is passed to the worker to flip the switch, and a clever worker-in-a-string trick is used to keep everything in one fiddle (hence the use of /* */ code comments over //). The same worker can be reused for any number of transforms, which scales well.

This remains an imperfect API however, requiring applications to know which bits to leave alone to not upset the codec-specific packetizer, with some codecs tougher to deal with than others — looking at you H.264 — which bits are they? Trial and error is required.

These changes are also not negotiated with the peer, something that is still being ironed out in the Working Group. But as of right now, this works in all browsers for any app that wants it, so that’s something!

The shim

The standards-track API is implemented natively in Safari and Firefox. It might not be obvious in the example — JSFiddle hides plug-ins well — but it relies on a small shim in Chrome. Also, two minor workarounds highlighted by the comment /* needed by chrome shim: */ were too hard to shim (like onrtctransform in a worker). Some of these are being fixed in Chromium already, so they may not be needed for long. Implementations should converge soon, which means this shim will hopefully be short-lived.

What about main-thread use cases?

There’s a WebRTC Samples example called Video Analyzer that dumps some metrics on the page. As of this writing, it is still written the old way, blocking video frame delivery on main-thread even though frames aren’t being modified. Jank may not be an issue in such a simple demo, but it still encourages developers to follow a pattern likely to scale poorly, and it also won’t work in Firefox or Safari. The pull-request webrtc/samples#1646 updates it to use the standards-track API with the shim mentioned earlier which may end up in adapter.js. Try it here in all browsers:

  1. Click Start!, share your camera, and then press Call
  2. .

The updated example shows video-frame delivery didn’t need to block on main-thread in the first place. Instead, simple worker messages post to main-thread to update the visible counters, which seems cleaner and more efficient.

Any application that needs the encoded frames on main-thread can still get them by using transferable streams. The burden of transfer is merely reversed.

Hopefully this blog post helps demystify the use of workers with this API! The need for a shim should hopefully diminish soon. In the meantime, I hope you’re able to put this API to good use, and feel free to ask any questions. There’s no comments section here, but you can reach me on twitter.

November 8, 2019

Removing Old Versions of DTLS

Contributed by Nils Ohlmeier, Hacking on real time communications since 2002

As you probably have read already old versions of TLS are going to be removed in March 2020. Obviously we don’t want to leave Firefox WebRTC users behind and vulnerable. Thus we are planing on disabling support for DTLS 1.0 also in March 2020.

According to our measurements the absolute majority (98.12%) of WebRTC services use DTLS 1.2 already today.

WebRTC DTLS usage numbers

The remaining 1.88% need to start upgrading to DTLS 1.2 today.

To help with the transition and testing we added new user preferences to Firefox 71 (currently available as Firefox Beta):

  • media.peerconnection.dtls.version.min = 770
  • media.peerconnection.dtls.version.max = 771

To test with DTLS 1.2 only simply open about:config and set media.peerconnection.dtls.version.min to 771. From now on your Firefox will no longer offer DTLS 1.0, as if it would be a release from March 2020.

July 1, 2019

Camera & microphone require https in Firefox 68.

Contributed by Jan-Ivar Bruaroey,

As of Firefox 68—releasing next week—camera and microphone will require an https connection to work. Access from insecure http will cease to work, matching how Chrome works. Camera and microhone are powerful features, and Firefox will now only expose them in secure contexts, a security and privacy improvement.

Largely an issue for developers to catch up on, this may nonetheless affect end users who follow old http links to servers that either still don’t redirect their users to https automatically, or don’t support it at all. If you’re experiencing this problem, click your URL bar, and if the URL starts with http, try changing it to https, and it should work again, provided the server supports https.

The rest of this blog post is for web developers, since there are some details that may be of interest to them. Firefox is deprecating camera and microphone access in insecure contexts in two steps: (more…)

April 7, 2019

Perfect negotiation in WebRTC

Contributed by Jan-Ivar Bruaroey,

New preface: What if you could add and remove media to and from a live WebRTC connection, without having to worry about state, glare (signaling collisions), role (what side you’re on), or what condition the connection is in? You’d simply call pc.addTrack(track, stream) regardless of time and place, and your track would just show up on the opposite side, without risk of terminal connection failure. A pipe dream? Too much to ask? Actually, now that Chrome has finally fixed its negotiationneeded event, this almost works! But you’re not using it, because “almost” doesn’t cut it. It only works 95% of the time, and the stakes from glare are too high in Chrome (no way to roll back from it, just pc.close()). But this original promise of the API is still within reach. We just need browsers other than Firefox to implement rollback, and fix some glaring (no pun intended) races in a few methods in the specification.

Perhaps few things in the WebRTC API cause as many pilot errors as the asymmetric exchange of SDP and ICE. Its stateful signalingState and timing sensitive ICE trickling can be a source of races if programmed incorrectly. As if that’s not challenging enough, keeping the two sides in sync and the two directions apart throws a lot of people for a loop (more puns).

What if I told you we can tame this complexity? That the manual way most people go about negotiating in WebRTC today is inferior, and possibly racy?

I’m talking about negotiationneeded and "rollback". Two mechanisms that can be used in concert to abstract negotiation away entirely. Yet, no-one is using them, because they haven’t worked, and don’t work yet in Chrome, respectively. We’ll introduce both. (more…)

August 6, 2018

isRemote in getStats() will disappear in Firefox 66

Contributed by Jan-Ivar Bruaroey,

Are you using getStats() for remote statistics in Firefox? Look for this warning today in Firefox Nightly (63, not 66):

⚠ Detected soon-to-break getStats() use! stat.isRemote goes away in Firefox 66, but won’t warn there!

TL;DR: If you see this warning in web console, read on to act now. Firefox 66 will move remote stats to "remote-outbound-rtp" and "remote-inbound-rtp", and isRemote will disappear, to comply with a change to the spec.

Firefox 63-65 will warn about this, but for technical reasons Firefox 66 will not be able to. This is because once the stats move in 66, there’s really no way to detect that you intended to look for remote stats the old way.

Update: The original article said Firefox 55. We’ve pushed it back to Firefox 66, giving you a little more time!

(more…)

July 12, 2018

getDisplayMedia now available in adapter.js

Contributed by Philipp Hancke, doing things webrtc at appear.in

If you ever had a meeting over video and wanted to present some slides, there is a high chance you have used screen-sharing to do so. The WebRTC specification recently converged on a standard way to accomplish this. It took a fairly long time, because the security considerations for a web page accessing the pixels of your entire screen or another window are quite serious.

Browsers are actively implementing the standard getDisplayMedia API now, with Microsoft Edge being the first to ship a native implementation. You can track the current implementation status for Firefox, Chrome, Microsoft Edge and Safari.

Both Chrome and Firefox have long supported screen-sharing using slightly different and non-standard APIs. A couple of weeks back, Harald Alvestrand at Google asked whether it was possible to polyfill navigator.mediaDevices.getDisplayMedia for screen-sharing in adapter.js.

Update 11/17/18: getDisplayMedia used to live on navigator but was recently moved to navigator.mediaDevices.getDisplayMedia. Examples and adapter.js have been updated to match.

(more…)

July 2, 2018

How to avoid Data Channel breaking

Contributed by Nils Ohlmeier, Hacking on real time communications since 2002

All the browser with support for negotiating data channels via SDP are using the same format. But the format is based on a specification from 2013. Since the specification has changed a lot since then it’s time to update the implementations to meet the latest spec.

Current status

If you are negotiating a data channel today in Firefox 61 (or current Chrome release) the SDP will looks something this:

(more…)

July 2, 2018

New Tool for Debugging WebRTC

Contributed by Michael Froman, crossing IP communication streams since 2005

Debugging WebRTC? Ever wanted a bit more visibility into the flow of a WebRTC call?  Or see exactly what PeerConnection API calls were made and when? Have we got a debugging deal (dev tools plugin) for you! Go here or search for “dev tools media panel” on about:addons.

Getting Started

After loading the plugin and starting a call on, for example, appear.in, open the dev tools (Tools -> Web Developer -> Toggle Tools).  Next, click on the “Media-Webrtc” pane.  The “Media-Webrtc” pane is most likely at the far right. Finally, selecting the Webrtc tab shows something like:
(more…)

May 22, 2018

Firefox is now supported by Google Hangouts and Meet

Contributed by Nils Ohlmeier, Hacking on real time communications since 2002

After extensive work from the Google Hangouts team and the Firefox WebRTC team both, the consumer version Google Hangouts and the enterprise version Google Meet, are working in Firefox with no plugin required thanks to WebRTC!

How we got here

Turning off the NPAPI  support in Firefox 53 resulted in the Google Hangouts plugin not working anymore. Unfortunately that meant that Firefox users could no longer enjoy Google Hangouts or Meet.

Google Chrome users could continue to use Hangouts and Meet without a NPAPI plugin, because the Google Hangout backend service had support for a flavor of WebRTC only implemented in Google Chrome. But that version was not spec compliant and was thus never supported by Firefox.

It took a considerable amount of effort from the Google Hangouts team to update their backend service to the latest version of the WebRTC specs to support Firefox. At the same time the Firefox WebRTC team implemented a bunch of features to support large scale conferencing for Google Meet.

In fact Google Hangouts, the consumer version, started working with Firefox 56. Now with Firefox 60 we were able to ship all the required features to also support Google Meet.

Thanks to all the hard working people involved in this project and resulting in making the Open Web a better and safer place from today on!