JSRuntime is now officially single-threaded

Given this title, a reasonable reaction would be:

Wait, wait, single threaded?!  But isn’t that, like, the wrong direction for the multicore present and manycore future?

so let me start by clearing this up:

A single SpiderMonkey runtime (that is, instance of JSRuntime) — and all the objects, strings and contexts associated with it — may only be accessed by a single thread at any given time. However, a SpiderMonkey embedding may create multiple runtimes in the same process (each of which may be accessed by a different thread).

That means it is up to the embedding to provide communication (if any) between the runtimes via JSNative or other SpiderMonkey hooks. One working example is the new implementation of web workers in Firefox which uses a runtime per worker. Niko Matsakis is experimenting with a different architecture in his new parallel JS project.

So that’s the quick summary. Now, for the interested, I’ll back up and explain the situation, how we got here, and where we are going in more detail.

Ghosts of SpiderMonkey past

In the beginning, as Brendan explains, Java-style big-shared-mutable-heap concurrency was all the rage and so, as Java’s kid brother, SpiderMonkey also had big-shared-mutable-heap concurrency. Now, locks weren’t ever (afaik) exposed to JS as part of SpiderMonkey, but an embedding could add them easily with a JSNative. However, SpiderMonkey did support concurrent atomic operations on objects with a clever (patented, even) locking scheme that avoided synchronization overhead for most operations.

This initial threading design stayed in place until about a year before Firefox 4.0 when the compartments project picked up steam. The key new concept introduced by this project was, well, the compartment. A runtime contains a set of compartments and each compartment contains a set of objects. Every object is in exactly one compartment and any reference between objects in different compartments must go through a wrapper. With compartments and wrappers, you can implement a sort of membrane that is useful for all kinds of things: GC, security boundaries, JIT compilation invariants, memory accounting, and JS proxies. Overall, I would say that compartments are one honking great idea.

The important thing about compartments for this story, though, is that the implementation effort really wanted single-threaded access to everything in a compartment. To be honest, I don’t know the particular technical issue raised at the time, but it isn’t hard to see how single-threaded-ness was a necessary simplification for such a challenging plan (viz., shipping compartments with Firefox 4). Anyway, the decision was made and compartments became single-threaded.

After Firefox 4 another great choice was made to rewrite Firefox’s implementation of web workers to not use XPConnect and to instead create a new runtime per worker. The choice was made because, even though a runtime allowed multi-threaded execution, there were still some global bottlenecks such as GC and allocation that were killing workers’ parallelism.

I’m Talking About Drawing a Line in the Sand

With web workers in separate runtimes, there were no significant multi-threaded runtime uses remaining. Furthermore, to achieve single-threaded compartments, the platform features that allowed JS to easily ship a closure off to another thread had been removed since closures fundamentally carry with them a reference to their original enclosing scope. Even non-Mozilla SpiderMonkey embeddings had reportedly experienced problems that pushed them toward a similar shared-nothing design. Thus, there was little reason to maintain the non-trivial complexity caused by multi-threading support.

There are a lot of things that “would be nice” but what pushed us over the edge is that a single-threaded runtime allows us to hoist a lot data currently stored per-compartment into the runtime. This provides immediate memory savings and also enables another big change we want to make that would create a lot more compartments (and thus needs compartments to be lighter-weight).

Thus, the decision was made to try to make SpiderMonkey single-threaded as an API requirement. A bug was filed in April 2011 and an announcement was made on dev.tech.js-engine a month after.

Across this line you do not…

April 2011 to… January 2012… what took so long?

Well, to begin with, there were quite a few minor uses of JSRuntime off the main thread that had to be chased down. Also, each of these cases required understanding new parts of the codebase and, in several cases, waiting a few months for other kind-hearted, but very busy, Mozillians to fix things for me. The biggest problem was xpcom proxies (not to be confused with JS proxies, which are awesome). Fortunately, Benjamin Smedberg already had a beef with xpcom/proxy and (just recently) nuked the whole directory from orbit.

After getting try server to pass without hitting any of the 10,000 places where single-thread-ness gets verified in debug builds, we couldn’t exactly just rip out the multi-threading support. The worry we had was that some popular add-ons would break the whole browser and we’d be faced with an uncomfortable backout situation. Thus, we landed a simple patch that asserts single-threaded-ness in a few pinch points in release builds and waited for the assert to make its way to a bigger audience. (I think this is a great example of how the rapid-release process enables developers.)

As of right now, the assert is in Firefox 10 Beta and slated to be released on January 31st. There are three four known offending extensions:

  • The IcedTea Java plugin on Linux seems to hit the assert for some applets. [Update: this was reported fixed in version 1.2pre]
  • BExternal.dll and gemgecko.dll are touching the main-thread only pref service off the main thread (already a bug) which ends up calling a JS observer. [Update: Both Gemius and Babylon seem to have shipped fixes]
  • [UPDATE] The DivX plugin.

Based on these results, we are concluding that the invariant “stuck” and thus we can actually make changes that assume single-threaded-ness. Indeed, the first bomb has been dropped (taking along 2200 lines of code along with it, and this is just the beginning).

The single-threaded invariant in detail

Each runtime now has an “owner thread” (JSRuntime::ownerThread). The “owner thread” is the id of the only OS thread allowed to touch the runtime. This owner thread is set to the current thread (PR_GetCurrentThread()) from JS_NewRuntime and may only be changed — when no JS is executing in the runtime — via JS_ClearRuntimeThread/JS_SetRuntimeThread. Virtually every JSAPI function that takes a JSContext parameter will assert that PR_GetCurrentThread == cx->runtime->ownerThread().

It should be mentioned that there are still a few remaining sources of concurrency:

  • The background-sweeping thread cleans up garbage objects which don’t have finalizers. Its interaction with the VM is pretty well isolated to the GC.
  • Pretty much the only JSAPI function that can be called off the main thread is JS_TriggerOperationCallback. This is how the the watchdog thread stops runaway JS. Fortunately, the interaction with the VM is through a single field: JSRuntime::interrupt.

One last thing to point out is that SpiderMonkey’s architecture will likely continue evolving to meet new concurrency needs. Indeed, concurrent runtimes may one day return in a more restricted and structured form. But maybe not; we’ll see.

9 Responses to JSRuntime is now officially single-threaded

  1. Pingback: Multithreading in scripting languages « Chris Dragan's Corner