Module scoping and linking

The ECMAScript TC39 committee had its regular meeting this week. One of the things we talked about were some revisions that Sam Tobin-Hochstadt and I have been discussing for the Harmony module system.

In previous iterations, the design allowed nested modules to refer outside their bodies only to other modules. For example:

    module A {
        var foo = 42;
        module B { ... }
    }

Module B would be allowed to refer to A, but not foo. This would also be true if module B could be loaded from an external file:

    module A {
        var foo = 42;
        module B = load "b.js";
    }

On es-discuss, Jasvir Nagra and Ihab Awad raised the concern that this design renders externally-loaded modules too sensitive to external scopes, similar to textual inclusion (aka #include). To be fair, they would only be sensitive to modules in scope, since other bindings would be removed from scope. But the point still stands: non-local changes to the lexical environment in external files would affect externally loaded modules. It’s not too hard to come up with plausible scenarios where this could bite programmers.

The revision we discussed proposes separating the cases of externally loaded modules and lexically nested modules. For one thing, in the first example above, why shouldn’t B be able to refer to foo? It’s right there! That’s just good old lexical scope. But if B is loaded externally, it should not be sensitive to a lexical environment of unbounded size. But at the same time, we still need to provide the ability to link together modules, even with cyclic dependencies, and place them in separate files. For example, an MVC app might be divided up into a few sub-modules:

    module MyApp {
        ...
        module Model = load "model.js";
        module View = load "view.js";
        module Controller = load "controller.js";
        ...
    }

The individual sub-modules ought to be able to import from one another, even cyclically — for example, the view and the controller might be mutually dependent. In essence, MyApp is creating a local module graph, consisting of its immediately nested module declarations.

So instead of inheriting the entire lexical environment of the code loading them, the modules Model, View, and Controller should be given a lexical environment with just the modules bound in the local module graph. That way, within every component of a program, you can easily divide it up into separate pieces, each of which may depend on other pieces. But at the same time, because they only see the other modules nested within MyApp, the sub-modules aren’t sensitive to code changes in the rest of the program.

10 Responses to Module scoping and linking

  1. Pingback: Tweets that mention Module scoping and linking « dherman at mozilla -- Topsy.com

  2. “… should be given a lexical environment with just the modules bound in the local module graph.”

    In that example the only bindings in MyApp are module bindings, but (just to be super-clear) if there were any let, const, var, or function declarations in MyApp (and only in MyApp), those declarations too would be in scope for Model, View, and Controller.

    IOW the only bindings visible to the loaded modules are the ones declared directly in the loading module, and not any in outer, statically containing modules.

    /be

  3. My comment is wrong — the only bindings in scope from the loading module for the loaded modules are module bindings. This looks like the minimal solution that keeps the simple implicit linking of the proposal and avoids capture problems.

    /be

  4. The idea really is that only the local modules are in scope for externally loaded modules, and no other bindings. This avoids the #include hazard of being sensitive to too many bindings from an entirely different file, and also ensures that in any given file, and free references must only be references to modules.

    Dave

  5. so, given MyApp nests 3 modules, the only way to make Controller and View inter-operate are calling binding functions inside MyApp context.

    personally i like this approach,

  6. @sombriks: It’s a little easier than that: Controller and View can refer to each other by name and import each other’s bindings. For example, if (say) the controller wants to send status updates to the view, it can import a “showStatus” function from View:

    // controller.js
    import View.showStatus;

    showStatus(currentStatus);

    In other words, the external files can refer to modules in the local module graph by name (in this example, inside an import declaration). But other than that they can’t refer to anything in the scope of MyApp.

    Dave

  7. What do you think of Newspeak’s modules[1], in this connection? In particular, the idea of modules being more module *factories*, with the environment for the module being passed in.

    [1] G. Bracha, P. von der Ahé, V. Bykov, Y. Kashai, W. Maddox, and E. Miranda, “Modules as Objects in Newspeak,” Design, Springer Verlag LNCS, 2010.

  8. Tony,

    Well aware of Newspeak’s approach, and I very much applaud the continuing module system research in the PL world.

    However, explicitly parameterized modules are something we’ve worked hard to avoid, because they have historically suffered from usability and complexity issues. We’ve worked to strike a balance between being able to allow users to link up their modules fairly flexibly while at the same time maintaining programmability with a lightweight semantics that allows you to get your code up and running right away.

    Dave

  9. Pingback: Module scoping and linking « dherman at mozilla - Rahsia Jana Wang Mudah

  10. (Don’t mind me jumping on the pile:)

    Explicitly parameterized first-class modules are arguably already “in” JS, especially in ES5. JS has objects (which can be frozen now) including first-class functions.

    In contrast, Simple Modules are second class until you ask for a reflection, which makes them easier to use as well as complementary to existing features in JS.

    /be