Large Data Channel Messages
Until very recently you had to be careful about the maximum message size you could send over a WebRTC data channel. Especially if the data channel had been established between Firefox and Chrome. See here for a very good explanation of the problem.
The good news is that bug 979417, a big contribution by Lennart Grahl, has landed in Firefox 57, which adds EOR handling to Firefox data channels.
EOR… what?
SCTP is the transport protocol used for exchanging WebRTC data channel messages. Its API provides an EOR flag which stands for end of record and tells us whether a message is partial or complete. Thus, every implementation needs to pay attention to this flag or message integrity will be violated. This is sadly the case for all popular data channel implementations at the moment and results in harsh size limitations for data channel messages. For more technical information on this and how one can work around this, take a look at Lennart’s previous blog posting.
Unfortunately, none of the browsers today support the SCTP ndata extension yet, so a single data channel message will monopolise the whole SCTP association. This phenomenon is also known as head-of-line blocking and makes sending large messages quite painful if other channels need to keep exchanging messages frequently, for example in a real-time chat with file-sending capability. All messages exchanged in the chat would arrive after the lengthy process of sending a gigantic file has been completed.
While the ndata extension is currently in the specification process, it’s making good progress and there’s no excuse for holding back proper EOR handling anymore. Lennart had gained experience in writing a standalone data channel implementation with EOR handling when he implemented RAWRTC. He decided that it was about time to add support to at least one of the browsers.
Exchanging Large Messages
With EOR handling, one major benefit is of course being able to send and receive large messages. But hold on a second… sending large messages from Firefox to Firefox was possible before, wasn’t it? Yep, that’s true, with help of the deprecated PPID-based fragmentation/reassembly mechanism. However, no one else implemented this. A detection algorithm has been added, so Firefox will fall back to use PPID-based fragmentation/reassembly when needed: in case the maximum message size attribute in the remote SDP is missing and 256 incoming streams have been negotiated for the SCTP association, it is assumed that the other peer is an older Firefox browser. It is not the cleanest of solutions but it should be adequate and will be removed after Firefox ESR has EOR support.
Thanks to the new EOR feature Firefox now supports receiving up to 1 GiB messages. Why is there a limitation? Well, theoretically we can support much more but the limiting factor is available memory and the fact that the RTCDataChannel API of WebRTC is too high-level, meaning it’s not possible to send or receive a message in chunks. Therefore, when you want to send a large file as a whole, there’s an enormous backpressure due to the fact that the file’s data needs to be in memory when calling the send method. Changing this requires changes to the W3C WebRTC specification.
Nevertheless, this size limitation may be increased in the future. But having a size limit requires knowledge about this limitation. More precisely, that value must be available before an application even tries to send a message or it would be a guessing game…
Maximum Message Size Indication
In the W3C WebRTC spec, there’s an RTCSctpTransport object which has a field called maxMessageSize. It maps to the attribute from the SDP and is exactly what a user application needs to look for. So, the following example should work, right?
let pc = new RTCPeerConnection(...);
let dc = pc.createDataChannel('test', {
negotiated: true,
id: 1
});
dc.onopen = () => {
let maximumMessageSize = pc.sctp.maxMessageSize;
dc.send(new Uint8Array(maximumMessageSize));
};
// Create and exchange offer/answer here.
// ...
But sadly, it doesn’t because Firefox and all other browsers currently don’t support the object API yet, that has been borrowed from the ORTC specification. So, what can be done to retrieve this information? The answer lies within the SDP blob. Check this out:
let pc = new RTCPeerConnection(...);
let maximumMessageSize;
let dc = pc.createDataChannel('test', {
negotiated: true,
id: 1
});
dc.onopen = () => {
dc.send(new Uint8Array(maximumMessageSize));
};
mySignaling.onRemoteDescription = (description) => {
// Parse a=max-message-size
maximumMessageSize = 65535;
const match = description.sdp.match(/a=max-message-size:\s*(\d+)/);
if (match !== null && match.length >= 2) {
maximumMessageSize = parseInt(match[1]);
}
// Set remote description
...
};
// Create and exchange offer/answer here.
// ...
Now we know how much data we can send. In case an application sends a message that exceeds this limitation, the send method will raise an error as described in the next section. But you probably don’t want to do the SDP inspection yourselves, so support in adapter.js has been created. We encourage you to update to the latest adapter version once that PR has been merged as it also sorts out various other incompatibility nits regarding the maximum message size for many browser interoperability cases.
Explicit Errors
These changes will not only make it possible to send and receive large data channel messages, but will also let you know if the other peer is capable of receiving a message that has been passed to the send method in form of a TypeError exception as specified (formerly, the data channel has been closed implicitly in case of such an error). This ensures backwards compatibility to implementations that do not handle EOR at the moment as all implementations are capable of handling 64 KiB sized messages.
Thank you to Lennart Grahl for helping writing this blog post and especially for contributing the patch to Firefox in the first place!
