Discussion Forums Redux: Choosing a Path Forward

mburns

7

Over the last several months I have collected feedback about how to improve the Mozilla Discussion Forums from MozCamp Europe, the Mozilla Blog, from newsgroup threads on Mozilla.General as well as direct feedback from real live people. The feedback echoed problems we have heard before, and confirmed that the current state of mailing lists at Mozilla is found wanting by many in the community. To that end, we have come up with some possible solutions to address these shortcomings. Given our unique size and requirements, no one solution is a magic bullet, each has its own benefits and drawbacks, and we need help deciding the correct path to take.

What are we fighting for?

Our current Mailman-based Forum system isn’t perfect, and we have come to know some of its warts all too well. We have previously collected complaints and desired features that the community would like to see resolved. Highlights include:

  • Posting via the Web. Currently this is done through Google Groups, would be nice to be done ‘in house’, so that we could customize, and optimize the interaction.
  • Relying on Google Groups has caused painpoints when problems arise, particularly because Mozilla has limited access into solving those problems.
  • Cumbersome management for moderators to deal with incoming message SPAM.

Whichever path forward is chosen, Mozilla will continue

What does great success look like?

An upgraded Discussion Forum system should improve the interaction of the website interface (that users expect to exist), whether by stabilizing our Google Groups integration, or using a different software setup that can provide a native web interface to both read and post to the Forum. An accessible Web UI to the Forums could breath new life into lists and projects around Mozilla. To manage the (new and existing) users and their messages, improved SPAM and moderation management would also be greatly welcome by list Moderators.

So with all that in mind, here are several high level(-ish) options we can pursue:

1. Keep on, keepin’ on

One (not very glamorous) option is to not significantly change anything. We can continue to rely on Google Groups to provide the sole web-accessible route to post to the Mozilla Discussion Forums via GigaNews providing NNTP mirroring of our mailing lists. This avoids achieving any of the big goals, but is comparatively lowest impact.

There are a few outstanding bugs that have been low priority or considered disruptive and thus haven’t found their way to getting Resolved:Fixed yet. Among them, odd message threading behavior that can make reading the Forums from e-mail clients like Thunderbird more cumbersome than it should be. These are getting addressed, and will expect to still be fixable if we moved to another system.

On the other hand, the conservative approach doesn’t address long-standing issues like providing a (more) reliable interface to post via the Web . While we have plans to improve SPAM protection, it requires a bit of duck tape and bailing wire to pull it all together.

2. Same same, but different

We’ve used Mailman 2 for about 10 years now, and version 3.0 has promised a number of useful features that might serve as a basis for achieving our feature requests. Its most promising feature (to us) is providing a native web interface for posting to lists (not just reading the archives), but that is still just a glimmer in the development team’s eye. The groundwork was laid, but the code has not yet been written to actually do it. Mozilla has an opportunity to organize development down this road, be it through Summer of Code internships, or sounding the horn across Mozilla community of developers to organize around this or even by providing a bounty.

In particular, Mailman 3.0 has a few other under-the-hood changes that make it an appealing upgrade path. A comprehensive REST API (finally!) means more common tasks (requesting a project mailing list) can be automated away. In fact, that API is the basis for a future Web interface for posting messages. Version 3 also manages subscriptions site-wide, using Persona, which eliminates monthly reminders.There has even been some work in adding native NNTP directly to Mailman. Multiple domain support would let us potentially combine the FOUR separate Mailman instances Mozilla uses to support all our various needs in our consolidated configuration.

Even with a Mailman 3 upgrade, there is no definitive date for supporting web-based posting to the forums. The API exists, developers have talked about it being a pretty neat idea, but it hasn’t begun actual development. But we could be the spark.

3. Starting over from scratch

While Mailman is certainly a popular listserv, it is far from the only one. Several similar Open Source projects exist that offer a similar-but-not-identical featuresets and have not been hindered by backwards compatibility concerns for the last several years. Among them, GroupServer and Nabble, which both provide a native web interface for posting, along with email access to lists as expected, however, there is no clear way to support NNTP, so Newsgroup access would break Google Groups and GigaNews support. Some light sampling of Forum posts showed upwards of 30% of posts to all Mozilla Newsgroups come from News, given the option. Either dropping NNTP, or trying to spin up some sort of proxy to sync to GigaNews, is not (necessarily) out of the question.

4. And now for something completely different

Then again, maybe Mozilla should not try to be all things in all cases by running services in-house when dedicated experts might do better. Using a hosted service could provide a straightforward Web Forum + Email list integration with a (potentially) more actively maintained interface. Effectively, this would mean dropping the niche access method (NNTP), and doubling down on the two more common methods (Web + Email). The drastic improvements possible to a well integrated web interface for new and experienced users could justify this option.

Because we have not (as of yet) found a commercial product that matches our feature requirements (which ideally include native NNTP integration) that can operate at Mozilla’s scale, certain feature requirements would need to be dropped, and NNTP is the lowest hanging fruit on that list.

We hack, you decide

Again, it is important to point out that there is no one magic bullet for solving all known complaints in any Discussion Forums setup, let alone one that must accomidate a community of thousands of developers and millions of users. We can only hope to reasonably approximate our ideal setup.

If you have avenues that haven’t been listed above, let us know. We want to make the best decision possible by considering all the possibilities ahead of us.

7 responses

  1. David Ross wrote on :

    For threaded discussions, I most definitely prefer NNTP. Web-based
    forums are just too cumbersome, not just Google Groups but also the
    forums hosted by various other software organizations (e.g., AVG,
    Adobe). And I feel that discussions conducted via E-mail distributions
    (e.g., listserv, Mailman) interfere with and even overwhelm “normal”
    E-mail.

    For the above reasons, I am disappointed by the pending reduction in
    support of Thunderbird from the Mozilla organization. I have been using
    Netscape 4, Mozilla Suite, and now Thunderbird as my newsreader.
    However, my interest in NNTP goes much further than news.mozilla.org; I
    am currently a member of the Big8 Management Board (see
    ).

    Thus, I feel that Mozilla should continue to provide a robust NNTP
    application and that NNTP should be the primary means of communication.

  2. WaltS wrote on :

    I have been using Thunderbird to participate in the NNTP support newsgroups, and
    find it quite annoying when I have to go to Google Groups to read an expired
    message, or to link to a previous post on a topic when composing a reply.

    I might as well get a Google account, use Google Groups, get my Email
    through Web Mail, and remove Thunderbird.

    It would not bother me if these two methods went away in favor of “something
    completely different” such as a Web Forum and Email.

  3. Sailfish wrote on ::

    Like WaltS, I also use NNTP support groups and here are some of my observations.

    NNTP:
    1. Way too many support groups as to making unclear on where best to post inquiries and, thus, encouraging cross-posting. Mozilla should cull the groups down to an imaginable number, perhaps one for each major product/project and then have the system allow a moderator to move a thread to a sub-group, if it’s warranted.

    2. Crowd-sourcing moderation has it’s own source of problems, not the least of which is the tendency for the crowd to get into a group-think mentality and then immediately attack a user, like antibodies, who may post a question that is critical, esoteric or seemingly impossible to reproduce. craigslist’s crowd-sourcing is a perfect example of this. Care should be taken on deciding to go this route over having a designated few moderators. Additionally, my preference would be to allow posts to become viewable immediately and then moderate them afterward in order to ensure a fluid nature in the discussion.

    3. Like the current NNTP newsgroups, you should allow one non-moderated group where off-topic discussions can flourish.

    4. Have a sophisticated and non-cumbersome search and refer-back system. The current inability (or lacking thereof) makes for much re-reporting of issues along with re-answering the problem. There should exist a way for the crowd to search for a prior question/answer to a new thread and then be able to post a link to the preferred answer, maybe some rating system like is used in StackExchange, e.g., http://math.stackexchange.com/questions/199676/what-are-imaginary-numbers

    5. The system should allow posting of inline screenshots to more easily clarify an issue.

    6. Only one moderator should have the ability to ban a subscriber.

  4. »Q« wrote on :

    In the peer support groups, I suspect the percentage of posts orginating through NNTP is much higher than 30%. All three of the three comments I see here so far were written by people active in the support groups. I believe that a solution that doesn’t include NNTP would harm the support groups, perhaps to the point of drying them up entirely.

    I don’t have time right now to analyze a sample of support group posts, but I could in a couple of weeks if numbers would be useful.

  5. The Wanderer wrote on :

    I also strongly prefer to retain the option of accessing the discussion forums by NNTP.

    All things considered, I would prefer either option 1 or option 2. The difference would hinge on what the layout and format of the Web interface suggested in option 2 would be, and specifically on whether it would be a traditional style of Web forum (with multiple message bodies per page); the reason for that is that the style and amount of quoting which are correct in such a Web interface is incompatible with the style and amount of quoting which are correct in a one-message-visible-at-a-time interface such as a newsreader or E-mail client.

    There was a discussion thread on rec.games.roguelike.angband in December 2007 on the topic of creating a crossposting interface between that newsgroup and a traditionally-styled, “flat”, multiple-message-bodies-per-page Web forum, with the Subject line “Newsgroup vs web forum”. I posted at length about the quoting-style conflict in that thread, and while I don’t have a link handy and don’t want to just copy-and-paste, I think what I said there would be applicable to this situation as well.

    If the concerns about incompatible “good quoting” styles can be resolved, I think option 2 would probably be best. Otherwise, I would strongly prefer to stick with option 1.

  6. justdave wrote on :

    Google Groups attempts to deal with the quoting issue by trying to detect quoted text and having all quotes collapsed by default, with a UI widget to expand them if you really want to see them.

  7. The Wanderer wrote on :

    @justdave: That works for the Google Groups interface – which is a Web interface – but doesn’t necessarily help people who use other interfaces. (I’ve seen the Google Groups quoting behavior cited as one of the several annoying things about Google Groups which Google refuses to fix, on the grounds that it’s a feature rather than a bug.)

    If the Web interface doesn’t quote automatically when replying, then someone who exclusively uses that interface is likely to not include quotes in their replies, which means that people on interfaces where quotes are appropriate won’t see them.

    If the Web interface *does* automatically include quoted text when replying. but doesn’t show it when the user is composing a reply, then someone who exclusively uses that interface will send the reply without ever seeing the quoted text – meaning they won’t snip it down, and people on interfaces where quotes are appropriate will see constantly expanding blocks of unsnipped quoted text. That’s not as bad as not having quotes at all on those interfaces, but it’s still nowhere near ideal.

    If the Web interface does automatically include quoted text when replying, and does show it when the user is composing a reply, then someone who exclusively uses that interface will have additional aggravation from having to deal with what may well seem like completely unnecessary quoted text; some people may decide to delete it entirely, since (from their perspective) there’s no benefit to including it. This might be a workable option if combined with some degree of “user education”, but if I put myself into the shoes of someone who does use the hypothetical Web inteface exclusively, I can certainly see why they might not like it.

    The only solution I can see offhand which seems like it might avoid most of the problems without introducing significant new ones would be if the hypothetical Web interface were designed to use a “one message displayed at a time” model, as is done on some of the mailing-list and-or newsgroup archive sites (e.g. perhaps Gmane, though that interface can be decidedly clunky). However, there’s no guarantee that the hypothetical Web inteface would indeed use such a design, and it’s possible that it would also have problems which I haven’t yet considered.