When should you display incoming WebRTC video?
The WebRTC specification has changed quite a bit since the first version in 2011. Jan-Ivar gave a great summary of how it has evolved over the years in a previous blog post.
One of the central questions for web developers is when to display an incoming video stream.
In its first iteration, the specification had an addstream event (since removed) which was triggered when a remote description was set. The addstream event had a stream object and you would set the video element’s srcObject property to that stream object like shown in this fiddle (which uses the more modern but similar track event):
For demonstration purposes, this fiddle does not complete a WebRTC connection between the two local objects. It never wires up the ICE candidates. The “Result” tab shows we get the track event, even though the connection never succeeds. In the early days of WebRTC, there were a lot of cases like this of “the remote video is a black box”. In particular, this happened when the peer-to-peer connection failed. Such as when a firewall is blocking something:
@janl @HenrikJoreteg 🙂 I have a slide about “things we’ve learned from @usetalky” that says, “Jan Lehnardt is always behind a firewall.”
— Adam Brault (@adambrault) April 11, 2014
In order to detect this, one has to listen for the iceconnectionstatechange event and evaluate it. The states demo shows how to listen for this state change. You would show the video when the iceConnectionState property of the peer connection changes to "connected" or "completed". This works quite well in simple cases where there is only a single stream.
With multiple streams, and when adding streams to a connection already in progress, this no longer works. The user might experience a very short period of blackness at the beginning of the new stream. Also, our video element now has to have access to the peer-connection state and that state it not even representing the right thing.
Track muted attribute to the rescue
In the latest iteration of the spec – the transceiver model that is implemented in Firefox 59 – the MediaStreamTrack has a muted attribute. This attribute is initially set to true and will change to false once media starts arriving. See the specification of the RTCRtpReceiver for details.
This allows for a much better decision on when to show the video element and when to hide it by listening for the unmute and mute events on the MediaStreamTrack as shown in this fiddle:
What we see in the logged output in the “Result” tab above, is that the track is initially muted and only becomes unmuted once video packets arrive, after the peer-to-peer connection becomes connected.
Unfortunately, this only works in Firefox 59 and up. There is a Chrome/WebRTC bug and similar issues exist in Microsoft Edge’s ORTC implementation. Hopefully the bugs around this get will fixed soon (the comments on the Chrome issue look positive), it is a small but crucial detail in the transition to WebRTC 1.0.
