July 12, 2018

getDisplayMedia now available in adapter.js

Contributed by Philipp Hancke, doing things webrtc at appear.in

If you ever had a meeting over video and wanted to present some slides, there is a high chance you have used screen-sharing to do so. The WebRTC specification recently converged on a standard way to accomplish this. It took a fairly long time, because the security considerations for a web page accessing the pixels of your entire screen or another window are quite serious. Browsers, however, are actively implementing the standard getDisplayMedia API now, with Microsoft Edge being the first to ship a native implementation.
You can track the current implementation status for Firefox, Chrome, Microsoft Edge and Safari.

Both Chrome and Firefox have long supported screen-sharing using slightly different and non-standard APIs.
A couple of weeks back, Harald Alvestrand at Google asked whether it was possible to polyfill navigator.getDisplayMedia for screen-sharing in adapter.js. My initial reaction was “no”, because it is rather complicated given how different the current implementations in Chrome and Firefox are. See here for a detailed explanation of some of the issues involved.

However, Jan-Ivar reminded me that one of the reasons we continue to invest time and effort into adapter.js is to help drive convergence towards the specification. This was compelling enough to give it a try. The result just shipped in adapter 6.3.0.

We did not want to make any default integration choices that break feature detection checking for ‘getDisplayMedia’ in window.navigator. Therefore, it does not do anything by default, instead requiring the developer to explicitly activate support and provide some details for the integration.

For Firefox, it requires you specify whether to present the option to share a screen or window to the user. E.g.:

if (adapter.browserDetails.browser == 'firefox') {
  adapter.browserShim.shimGetDisplayMedia(window, 'screen');
}

In the case of this example, calling navigator.getDisplayMedia({video: true}) will just work, asking the user to share a screen. As an intermediate step until getDisplayMedia we are discussing whether to merge the 'screen' and 'window' behavior to allow transparent shimming. Stay tuned for updates.

Doing a shim for Chrome is a bit more complicated. Chrome requires an extension for screen-sharing, and the screen/window picker is triggered from the extension background page. This requires some form of communication between the frontend javascript and the background page and there is no standardized way to accomplish this. The getscreenmedia library and Jitsi’s desktop sharing extension implements one way to do it using chrome.runtime.sendMessage. Other libraries or products like appear.in may use window.postMessage.

In order to accommodate any library in adapter.js, we opted for an non-opinionated approach which lets the developer supply a function that returns a Promise that resolves with the id of the screen/window or tab that the user has chosen. For extensions using getScreenMedia, the shim needs to activate the polyfill like this:

if (adapter.browserDetails.browser == 'chrome') {
  adapter.browserShim.shimGetDisplayMedia(window, function() {
    return new Promise((resolve, reject) => {
      if (!sessionStorage.getScreenMediaJSExtensionId) { // need to install extension
        var err = new Error('extension required for getDisplayMedia');
        err.name = 'ExtensionRequired'; // custom error name you need to check for later
        return Promise.reject(err);
      }
      chrome.runtime.sendMessage(sessionStorage.getScreenMediaJSExtensionId,
        {type: 'getScreen', id: 1}, null,
        function (data) {
          if (!data || data.sourceId == '') { // user canceled
            var error = new Error('NavigatorUserMediaError');
            error.name = 'NotAllowedError';
            reject(error);
          } else {
            resolve(data.sourceId);
          }
      })
    })
  })
}

This also handles the case where an extension is required by the shim, but the extension is not installed. In that case a custom error is thrown, to be handled by the calling code:

navigator.getDisplayMedia({video: true}).then(stream => {
  // do something with the stream
})
.catch(e => {
  if (e.name == myCustomErrorForExtensionNotInstalledString) {
    // come up with a UX for installation the extension
  }
});

Sadly the best way to get a decent user experience with inline installation will be deactivated in Chrome on September 12th, see the WebRTCHacks post on this.

After that, calling getDisplayMedia will just work, like it already does in Microsoft Edge:

navigator.getDisplayMedia({video: true}).then(stream => {
  // do something with the stream
})
.catch(e => {
  // handle any errors
});

Available now

We recommend feature detecting getDisplayMedia using 'getDisplayMedia' in window.navigator
until Safari ships getDisplayMedia, as well as to handle older versions of Microsoft Edge that do not support getDisplayMedia.

Compared to other parts of adapter.js, shimming getDisplayMedia is not as trivial to use (“just require it and it works”). Still, we hope it will pave the way for using getDisplayMedia compatibly, once (and even before) native implementations show up in browsers.

The implementation is an ugly workaround. However, it is better to hide these workarounds in adapter.js rather than dealing with the mess yourself. Or as Jan-Ivar says, we are driving convergence towards the specification. 🙂

July 2, 2018

How to avoid Data Channel breaking

Contributed by Nils Ohlmeier, Hacking on real time communications since 2002

All the browser with support for negotiating data channels via SDP are using the same format. But the format is based on a specification from 2013. Since the specification has changed a lot since then it’s time to update the implementations to meet the latest spec.

Current status

If you are negotiating a data channel today in Firefox 61 (or current Chrome release) the SDP will looks something this:

(more…)

July 2, 2018

New Tool for Debugging WebRTC

Contributed by Michael Froman, crossing IP communication streams since 2005

Debugging WebRTC? Ever wanted a bit more visibility into the flow of a WebRTC call?  Or see exactly what PeerConnection API calls were made and when? Have we got a debugging deal (dev tools plugin) for you! Go here or search for “dev tools media panel” on about:addons.

Getting Started

After loading the plugin and starting a call on, for example, appear.in, open the dev tools (Tools -> Web Developer -> Toggle Tools).  Next, click on the “Media-Webrtc” pane.  The “Media-Webrtc” pane is most likely at the far right. Finally, selecting the Webrtc tab shows something like:
(more…)

May 22, 2018

Firefox is now supported by Google Hangouts and Meet

Contributed by Nils Ohlmeier, Hacking on real time communications since 2002

After extensive work from the Google Hangouts team and the Firefox WebRTC team both, the consumer version Google Hangouts and the enterprise version Google Meet, are working in Firefox with no plugin required thanks to WebRTC!

How we got here

Turning off the NPAPI  support in Firefox 53 resulted in the Google Hangouts plugin not working anymore. Unfortunately that meant that Firefox users could no longer enjoy Google Hangouts or Meet.

Google Chrome users could continue to use Hangouts and Meet without a NPAPI plugin, because the Google Hangout backend service had support for a flavor of WebRTC only implemented in Google Chrome. But that version was not spec compliant and was thus never supported by Firefox.

It took a considerable amount of effort from the Google Hangouts team to update their backend service to the latest version of the WebRTC specs to support Firefox. At the same time the Firefox WebRTC team implemented a bunch of features to support large scale conferencing for Google Meet.

In fact Google Hangouts, the consumer version, started working with Firefox 56. Now with Firefox 60 we were able to ship all the required features to also support Google Meet.

Thanks to all the hard working people involved in this project and resulting in making the Open Web a better and safer place from today on!

May 9, 2018

Better privacy on camera mute in Firefox 60

Contributed by Jan-Ivar Bruaroey,

On a web conference call, have you ever wondered if people can see you? Maybe you’ve tried to decipher the site’s video-mute button, often some red camera symbol with a slash through it—“Is it on or off? Does red mean stop or live? Is the slash telling me I’m muted now, or to click here to mute?”—All the while the camera light at the top of your laptop keeps glaring at you brightly.

We have fixed this in Firefox 60.

We can’t fix the websites, but in Firefox 60, whenever you video-mute, your laptop or USB camera hardware light will now extinguish. Whenever you unmute again, it will light back up. No more worrying if you’re live! (more…)

April 4, 2018

Exploring RTCRtpTransceiver.

Contributed by Jan-Ivar Bruaroey,

Firefox now implements the RTCRtpTransceiver API, a.k.a. stage 3 in my blog post “The evolution of WebRTC 1.0.” from last year. Transceivers more accurately reflect the SDP-rooted network behaviors of an RTCPeerConnection. E.g. addTransceiver() (or addTrack) now creates a receiver at the same time, which correlates better with the bi-directional m-line that begins life in the SDP offer, rather than being something that arrives later once negotiation completes. This lets media arrive early on renegotiation.

But most WebRTC sites today are still written the old way, and there are subtle differences to trip over. Web developers are stuck in a bit of limbo, as other browsers haven’t caught up yet. In light of that, we should probably dive right into two immediate breaking spec changes you might have run into in Firefox 59:

  • Remote tracks are now muted and temporarily removed from their stream(s), rather than ended, in response to direction changes (e.g. from the poorly named removeTrack).
  • track.ids at the two end-points rarely correlate anymore.

Importantly, this affects you whether you use transceivers or not. addTrack (even addStream) is just a tweaked version of addTransceiver these days. This blog will look at how to handle this—tying up lose ends from last year’s blog post—and demonstrate how Firefox works. (more…)

November 3, 2017

Large Data Channel Messages

Contributed by Nils Ohlmeier, Hacking on real time communications since 2002

Until very recently you had to be careful about the maximum message size you could send over a WebRTC data channel. Especially if the data channel had been established between Firefox and Chrome. See here for a very good explanation of the problem.

The good news is that bug 979417, a big contribution by Lennart Grahl, has landed in Firefox 57, which adds EOR handling to Firefox data channels.

(more…)

September 30, 2017

When your video freezes

Contributed by Nils Ohlmeier, Hacking on real time communications since 2002

We recently fixed a regression in Firefox 57 that the max-fs parameter in fmtp lines had stopped working in Firefox 56. Because it got reported to us while Firefox 56 was still in it’s Beta cycle we uplifted the patch for Firefox 56 as well.

It got diagnosed a short time ago that this patch had unforeseen consequences: it can result in Firefox stopping to send video in the middle of a call.
(more…)