getDisplayMedia now available in adapter.js

If you ever had a meeting over video and wanted to present some slides, there is a high chance you have used screen-sharing to do so. The WebRTC specification recently converged on a standard way to accomplish this. It took a fairly long time, because the security considerations for a web page accessing the pixels of your entire screen or another window are quite serious.

Browsers are actively implementing the standard getDisplayMedia API now, with Microsoft Edge being the first to ship a native implementation. You can track the current implementation status for Firefox, Chrome, Microsoft Edge and Safari.

Both Chrome and Firefox have long supported screen-sharing using slightly different and non-standard APIs. A couple of weeks back, Harald Alvestrand at Google asked whether it was possible to polyfill navigator.mediaDevices.getDisplayMedia for screen-sharing in adapter.js.

Update 11/17/18: getDisplayMedia used to live on navigator but was recently moved to navigator.mediaDevices.getDisplayMedia. Examples and adapter.js have been updated to match.

My initial reaction was “no”, because it is rather complicated given how different the current implementations in Chrome and Firefox are. See here for a detailed explanation of some of the issues involved.

However, Jan-Ivar reminded me that one of the reasons we continue to invest time and effort into adapter.js is to help drive convergence towards the specification. This was compelling enough to give it a try. The result just shipped in adapter 6.3.0.

We did not want to make any default integration choices that break feature detection checking for ‘getDisplayMedia’ in window.navigator.mediaDevices. Therefore, it does not do anything by default, instead requiring the developer to explicitly activate support and provide some details for the integration.

For Firefox, it requires you specify whether to present the option to share a screen or window to the user. E.g.:

if (adapter.browserDetails.browser == 'firefox') {
  adapter.browserShim.shimGetDisplayMedia(window, 'screen');
}

In the case of this example, calling navigator.mediaDevices.getDisplayMedia({video: true}) will just work, asking the user to share a screen. As an intermediate step until getDisplayMedia we are discussing whether to merge the 'screen' and 'window' behavior to allow transparent shimming. Stay tuned for updates.

Doing a shim for Chrome is a bit more complicated. Chrome requires an extension for screen-sharing, and the screen/window picker is triggered from the extension background page. This requires some form of communication between the frontend javascript and the background page and there is no standardized way to accomplish this. The getscreenmedia library and Jitsi’s desktop sharing extension implements one way to do it using chrome.runtime.sendMessage. Other libraries or products like appear.in may use window.postMessage.

In order to accommodate any library in adapter.js, we opted for an non-opinionated approach which lets the developer supply a function that returns a Promise that resolves with the id of the screen/window or tab that the user has chosen. For extensions using getScreenMedia, the shim needs to activate the polyfill like this:

if (adapter.browserDetails.browser == 'chrome') {
  adapter.browserShim.shimGetDisplayMedia(window, function() {
    return new Promise((resolve, reject) => {
      if (!sessionStorage.getScreenMediaJSExtensionId) { // need to install extension
        var err = new Error('extension required for getDisplayMedia');
        err.name = 'ExtensionRequired'; // custom error name you need to check for later
        return Promise.reject(err);
      }
      chrome.runtime.sendMessage(sessionStorage.getScreenMediaJSExtensionId,
        {type: 'getScreen', id: 1}, null,
        function (data) {
          if (!data || data.sourceId == '') { // user canceled
            var error = new Error('NavigatorUserMediaError');
            error.name = 'NotAllowedError';
            reject(error);
          } else {
            resolve(data.sourceId);
          }
      })
    })
  })
}

This also handles the case where an extension is required by the shim, but the extension is not installed. In that case a custom error is thrown, to be handled by the calling code:

navigator.mediaDevices.getDisplayMedia({video: true}).then(stream => {
  // do something with the stream
})
.catch(e => {
  if (e.name == myCustomErrorForExtensionNotInstalledString) {
    // come up with a UX for installation the extension
  }
});

Sadly the best way to get a decent user experience with inline installation will be deactivated in Chrome on September 12th, see the WebRTCHacks post on this.

After that, calling getDisplayMedia will just work, like it already does in Microsoft Edge:

navigator.mediaDevices.getDisplayMedia({video: true}).then(stream => {
  // do something with the stream
})
.catch(e => {
  // handle any errors
});

Available now

We recommend feature detecting getDisplayMedia using 'mediaDevices' in window.navigator and 'getDisplayMedia' in window.navigator.mediaDevices until Safari ships getDisplayMedia, as well as to handle older versions of Microsoft Edge that do not support getDisplayMedia.

Compared to other parts of adapter.js, shimming getDisplayMedia is not as trivial to use (“just require it and it works”). Still, we hope it will pave the way for using getDisplayMedia compatibly, once (and even before) native implementations show up in browsers.

The implementation is an ugly workaround. However, it is better to hide these workarounds in adapter.js rather than dealing with the mess yourself. Or as Jan-Ivar says, we are driving convergence towards the specification. 🙂