Beer and Tell – October 2015

Once a month, web developers from across the Mozilla Project get together to forge Bitcoins. While we magnetize needles for carving out our counterfeit bits directly on hard drive platters, we find time to talk about our side projects and drink, an occurrence we like to call “Beer and Tell”.

There’s a wiki page available with a list of the presenters, as well as links to their presentation materials. There’s also a recording available courtesy of Air Mozilla.

ErikRose: Rubik’s Magic Cube

First up was ErikRose, who talked about the process of learning how to solve a Rubik’s Cube and how it affected his thinking. The learning process mirrored that of learning a language: first, you only see the cube as a block of individual colors, but as you progress you start to recognize specific cubes and arrangements of cubes as “words”, and eventually you become able to recognize abstract patterns of cubes relative to each other.

Check out the recording for a detailed walk through part of the cube-solving process!

Mythmon: N-Bodies Simulation in Rust

Next was Mythmon, who shared a physics simulation of bodies floating in space being affected by gravity. The simulation was written in Rust and relies on Piston for drawing graphics to the screen.

Potch: Canvas Blur

Potch was next with a demo of performing a Gaussian blur on a canvas in JavaScript. Branching off some experiments around auto-cropping algorithms, Potch’s demo processes the blur in chunks and produces results that are comparable to blurs produced in Photoshop, albeit much slower and with less sampling for the blur. While not intended to actually be used for anything, future improvements include using Web Workers to process the blur asynchronously and in parallel, as well as performing other convolutions besides blurring.

BWalker: ASCII Art Dashboard

Last up was bwalker, who demoed an ASCII art dashboard powered by blessed-contrib. The library provides the ability to make console-based dashboards with widgets for line graphs, bar charts, tables, and even a world map. This particular demo pulled some statistics from Github as well as graphing randomly generated numbers.


The highlight of this week’s meetup was lonnen’s impressive feat of creating 5 bitcoins on a flash drive using only a lighter and a chiropractic activator.

If you’re interested in attending the next Beer and Tell, sign up for the dev-webdev@lists.mozilla.org mailing list. An email is sent out a week beforehand with connection details. You could even add yourself to the wiki and show off your side-project!

See you next month!

Extravaganza – October 2015

Once a month, web developers from across Mozilla get together to see who can score the highest car emissions rating. While we scour the world for dangerous combustibles, we find time to talk about the work that we’ve shipped, share the libraries we’re working on, meet new folks, and talk about whatever else is on our minds. It’s the Webdev Extravaganza! The meeting is open to the public; you should stop by!

You can check out the wiki page that we use to organize the meeting, or view a recording of the meeting in Air Mozilla. Or just read on for a summary!

Shipping Celebration

The shipping celebration is for anything we finished and deployed in the past month, whether it be a brand new site, an upgrade to an existing one, or even a release of a library.

One and Done on Heroku

First up was bsilverberg with the news that One and Done, Mozilla’s contributor task board, has successfully migrated from the Mozilla-hosted Stackato PAAS to Heroku. One and Done takes advantage of a few interesting features of Heroku, such as App Pipelines and Review Apps.

Pontoon Sync Improvements and New Leaderboards

Next was Osmose (that’s me!) sharing a few new features on Pontoon, a site for submitting translations for Mozilla software:

  • mathjazz added a new “Latest Activity” column to project and locale listings.
  • jotes added time-based filters to the leaderboard as well as several performance improvements to the page.
  • I added inc file support to the new sync process, as well as a few other fixes such that all Pontoon projects are now using the new sync. Yay!

Air Mozilla iTunes Video Podcast

Peterbe stopped by to share the news that Air Mozilla now has an iTunes-compatible podcast feed for all of its videos. The feed has already been approved by Apple and is available on the iTunes Store.

DXR Static File Cachebusting

ErikRose shared the news that DXR now adds hashes to static file filenames so that updates to the static files don’t get messed up by an old local cache. Interestingly, instead of relying on popular tools like Grunt or Webpack, Erik opted to implement the hashing using some logic in a Makefile plus a little bit of Python. You can check out the pull request to see the details of the change.

Replacing localForage with localStorage

Next we went back to Peterbe, who recently replaced localForage, a library that abstracts several different methods of storing data locally in the browser, with localStorage, the blocking, built-in storage solution that ships with browsers. He also shared a blog post that showed that not only is localStorage simpler to use, it was actually faster in his specific use case.

Mozilla.org Hosted on Deis

Next was jgmize sharing the news that mozilla.org is now hosted on a Deis cluster. Currently a small amount of production data is being hosted by the cluster, but a larger rollout is planned for the near future. Giorgos is responsible for the entire Jenkins setup, including a neat deployment pipeline display, an Ansible playbook, and the deployment pipeline scripts. Pmac ported over hundreds of Apache redirects to Python and wrote a comprehensive set of tests for them as well.

Roundtable

The Roundtable is the home for discussions that don’t fit anywhere else.

Farewell to Wenzel

Lastly, I wanted to specifically call out wenzel, whose last day as a paid contributor for Mozilla was last Friday. Wenzel has been a Mozillian for 9 years, starting as an intern. He’s contributed to almost every major Mozilla web property, and will be missed.


This month’s winner was willkg, with an impressive 200% rating, generating more pollution than the fuel he put in. Local science expert Dr. Potch of Mr. Potch’s Questionable Ethics and Payday Loan Barn was quoted as saying that willkg’s score was “possible”.

If you’re interested in web development at Mozilla, or want to attend next month’s Extravaganza, subscribe to the dev-webdev@lists.mozilla.org mailing list to be notified of the next meeting, and maybe send a message introducing yourself. We’d love to meet you!

See you next month!

Using peep on Heroku

I recently moved a Django app to Heroku which was using peep, rather than pip, for package installation. By default Heroku will use pip to install your required packages when it sees a requirements.txt file in the root of your project, and the option to use peep instead does not exist.

Luckily one of my colleagues, pmac, was kind enough to create a share a Heroku buildpack which uses peep instead of pip for package installation. All you need to do to make use of this buildpack is issue the following command using the Heroku CLI for your app:

heroku buildpacks:set https://github.com/pmclanahan/heroku-buildpack-python-peep

Of course, make sure your requirements.txt is peep-compatible.

Happy peeping!

Beer and Tell – September 2015

Once a month, web developers from across the Mozilla Project get together to design user-friendly advertisements. When we’re not deciding which police alarm we want to auto-play, we find time to talk about our side projects and drink, an occurrence we like to call “Beer and Tell”.

There’s a wiki page available with a list of the presenters, as well as links to their presentation materials. There’s also a recording available courtesy of Air Mozilla.

Erik Rose: Whitespace-significant Meta-grammar for Homoiconic Languages

The only presenter this week was ErikRose, who shared a prototype grammar for a class of whitespace-significant programming languages. The goal was to represent the nested-list structure of homoiconic languages like Lisp while keeping redundant bracing out of the way like Python.

if is_yak then
    shave yak
    bathe yak
  else
    do productive_work

--> (if is_yak then
        ((shave yak)
         (bathe yak))
     else
        ((do productive_work))
    )

Normally, when the else statement in code similar to the sample above isn’t indented and is at the same level as the if statement, the parser must know that else statements may follow after if statements and are related to them. To get around this, ErikRose suggests using a partial outdent. This way, the parser can recognize that the else statement is separate from the shave and bathe lines but should still be grouped with the if statement, even if the parser doesn’t know what an if statement is.

Erik’s gist contains a few more samples and a possible grammar for parsing a language like this using Parsimonious, his parsing library.


Our focus group of QVC customers has given us great insight in to just what the average internet user is looking for in their sponsored content. Early retirement, here we come!

If you’re interested in attending the next Beer and Tell, sign up for the dev-webdev@lists.mozilla.org mailing list. An email is sent out a week beforehand with connection details. You could even add yourself to the wiki and show off your side-project!

See you next month!

Extravaganza – September 2015

Once a month, web developers from across Mozilla get together to to work on our containerization startup. While we buy up the latest stock of Tupperware, we find time to talk about the work that we’ve shipped, share the libraries we’re working on, meet new folks, and talk about whatever else is on our minds. It’s the Webdev Extravaganza! The meeting is open to the public; you should stop by!

You can check out the wiki page that we use to organize the meeting, or view a recording of the meeting in Air Mozilla. Or just read on for a summary!

Shipping Celebration

The shipping celebration is for anything we finished and deployed in the past month, whether it be a brand new site, an upgrade to an existing one, or even a release of a library.

Pontoon

Pontoon is a tool for translating Mozilla software. This past month saw several changes ship, including a brand new sync process that is up to 80% faster than the old sync process. Other changes include a new listing page for locale teams, a 50% speedup on the translation view, several new locales, and several visual improvements.

Crash-Stats Faster Loading

Crash-Stats is a service for analyzing crash reports from various Mozilla products. This month they shipped more aggressive cache headers for static assets, as well as Gzip compression for JavaScript, CSS, and HTML files. The result was a 25% improvement in load time! Further improvements are planned to reduce the time-to-first-byte.

DXR 2.0

DXR is a code search and navigation tool for Mozilla (and other) projects. DXR 2.0 has been in development for a long time and has shipped at last! The new version comes with a host of updates, including:

  • Support for C++ and Rust, as well as limited Python and XPDIL support
  • Parallel, clustered indexing
  • Binary file and image browsing
  • Plugin-based architecture
  • And more!

Open-source Citizenship

Here we talk about libraries we’re maintaining and what, if anything, we need help with for them.

Optisorl

Peterbe shared a library he created called optisorl. It is a pluggable backend for sorl-thumbnail that optimizes thumbnails using pngquant, gifsicle, and, coming soon, mozjpeg for optimizing the generated thumbnails.

Roundtable

The Roundtable is the home for discussions that don’t fit anywhere else.

Hacks Blog Redesign

Potch informed us of his in-progress redesign of the Hacks blog. A demo of his progress is available and he is looking for feedback. Check it out! Tell him how you feel about it!


If you sign up in the next month, you get a free upgrade to Rubbermaid-brand containers! As always, our compute instances are delivered straight to your door and are always fresh, never frozen.

If you’re interested in web development at Mozilla, or want to attend next month’s Extravaganza, subscribe to the dev-webdev@lists.mozilla.org mailing list to be notified of the next meeting, and maybe send a message introducing yourself. We’d love to meet you!

See you next month!

Node.js static file build steps in Python Heroku apps

I write a lot of webapps. I like to use Python for the backend, but most
frontend tools are written in Node.js. LESS gives me nicer style sheets, Babel
lets me write next-generation JavaScript, and NPM helps manage dependencies
nicely. As a result, most of my projects are polyglots that can be difficult to
deploy.

Modern workflows have already figured this out: Run all the tools. Most
READMEs I’ve written lately tend to look like this:

$ git clone https://github.example.com/foo/bar.git
$ cd git
$ pip install -r requirements.txt
$ npm install
$ gulp static-assets
$ python ./manage.py runserver

I like to deploy my projects using Heroku. They take care of the messy details
about deployment, but they don’t seem to support multi-language projects easily.
There are Python and Node buildpacks, but no clear way of combining the two.

Multi Buildpack

GitHub is littered with attempts to fix this by building new buildpacks.
The problem is they invariable fall out of compatibility with Heroku. I could
probably fix, but then I’d have to maintain them. I use Heroku to avoid
maintaining infrastructure; custom buildpacks are one step forward, but two
steps back.

Enter Multi Buildpack, which runs multiple buildpacks at once.

It is simple enough that it is unlike to fall out of compatibility. Heroku has a
fork of the project on their GitHub account, which implies that it will be
maintained in the future.

To configure the buildpack, first tell Heroku you want to use it:

$ heroku buildpacks:set https://github.com/heroku/heroku-buildpack-multi.git

Next, add a .buildpacks file to your project that lists the buildpacks to run:

https://github.com/heroku/heroku-buildpack-nodejs.git
https://github.com/heroku/heroku-buildpack-python.git

Buildpacks are executed in the order they’re listed in, allowing later
buildpacks to use the tools and scripts installed by earlier buildpacks.

The Problem With Python

There’s one problem: The Python buildpack moves files around, which makes it
incompatible with the way the Node buildpack installs commands. This means that
any asset compilation or minification done as a step of the Python buildpack
that depends on Node will fail.

The Python buildpack automatically detects a Django project and runs
./manage.py collectstatic. But the Node environment isn’t available, so this
fails. No static files get built.

There is a solution: bin/post_compile! If present in your repository, this
script will be run at the end of the build process. Because it runs outside of
the Python buildpack, commands installed by the Node buildpack are available and
will work correctly.

This trick works with any Python webapp, but lets use a Django project as an
example. I often use Django Pipeline for static asset compilation. Assets
are compiled using the command ./manage.py collectstatic, which, when properly
configured, will call all the Node commands.

#!/bin/bash
export PATH=/app/.heroku/node/bin:$PATH
./manage.py collectstatic --noinput

Alternatively, you could call Node tools like Gulp or Webpack directly.

In the case of Django Pipeline, it is also useful to disable the Python
buildpack from running collectstatic, since it will fail anyways. This is done
using an environment variable:

heroku config:set DISABLE_COLLECTSTATIC 1

Okay, so there is a little hack here. We still had to append the Node binary
folder to PATH. Pretend you didn’t see that! Or don’t, because you’ll need
to do it in your script too.

That’s it

To recap, this approach:

  1. Only uses buildpacks available from Heroku
  2. Supports any sort of Python and/or Node build steps
  3. Doesn’t require vendoring or pre-compiling any static assets

Woot!

Beer and Tell – August 2015

Once a month, web developers from across the Mozilla Project get together to spend an hour of overtime to honor our white-collar brethren at Amazon. As we watch our productivity fall, we find time to talk about our side projects and drink, an occurrence we like to call “Beer and Tell”.

There’s a wiki page available with a list of the presenters, as well as links to their presentation materials. There’s also a recording available courtesy of Air Mozilla.

openjck: Discord

openjck was up first and shared Discord, a Github webhook that scans pull requests for CSS compatibility issues. When it finds an issue, it leaves a comment on the offending line with a short description and which browsers are affected. The check is powered by doiuse, and projects can add a .doiuse file (using browserslist syntax) that specifies which browser versions they want to be tested against. Discord currently checks CSS and Stylus files.

The Discord team (mrmakeit, groovecoder, davidwalsh, and openjck) is looking for sites to test Discord out. Work on the site is currently suspended (which is why it’s a side project, the team may work on it in their free time) so that feedback can be gathered to determine where the site should go next. If you’re interested in trying out Discord, let groovecoder know!

peterbe: Activity and Fanout.io

Next up was peterbe, with an update to Activity. The site now uses Fanout.io and a message queue to improve how activity items are fetched from GitHub and other sources. The site queues up jobs to fetch data from the Github API, and as the jobs complete, they send their results to Fanout. Fanout’s JavaScript library maintains an open WebSocket with their service, and when Fanout receives the data from the completed jobs, it notifies the client of the new data, which gets written to localStorage and updates the React state. This allows Activity to remain structured as an offline-ready application while still receiving seamless updates if the user has an internet connection.


There’s a donation jar near the exit; for just $25 you can pay for an hour of time for an Amazon engineer to spend with their family. Checks may be made payable to No Questions Asked Laundry, LLC.

If you’re interested in attending the next Beer and Tell, sign up for the dev-webdev@lists.mozilla.org mailing list. An email is sent out a week beforehand with connection details. You could even add yourself to the wiki and show off your side-project!

See you next month!

Extravaganza – August 2015

Once a month, web developers from across Mozilla get together to recreate classic dishes using only soylent and food coloring. As we discover what oven temperatures are safe for the techno-wizard goop, we find time to talk about the work that we’ve shipped, share the libraries we’re working on, meet new folks, and talk about whatever else is on our minds. It’s the Webdev Extravaganza! The meeting is open to the public; you should stop by!

You can check out the wiki page that we use to organize the meeting, or view a recording of the meeting in Air Mozilla. Or just read on for a summary!

Shipping Celebration

The shipping celebration is for anything we finished and deployed in the past month, whether it be a brand new site, an upgrade to an existing one, or even a release of a library.

Pontoon: One L10n Platform to Rule Them All

First up was Osmose (that’s me!) sharing the news that Pontoon is now able to localize Firefox, Thunderbird, and other Gecko-based apps, making it a single tool capable of localizing every project across Mozilla! The Slovenian localization team is already using it for all the projects they actively translate, and more locales are signing up every day.

New Hires / Interns / Volunteers / Contributors

Here we introduce any newcomers to the Webdev group, including new employees, interns, volunteers, or any other form of contributor.

Farewell to new_one and nolski!

Intern season is winding down, and that means beloved Webdev interns new_one and nolski are leaving us! We wish them well and thank them for their hard work. New_one already gave a presentation on his contributions to DXR, and nolski’s presentation will be streamed on Air Mozilla on August 11th.

Roundtable

The Roundtable is the home for discussions that don’t fit anywhere else.

React Devtools in Firefox

Mythmon shared the news that the React Developer Tools extension has released a pre-release Firefox add-on. The extension allows developers working on React-based webapps to inspect the component hierarchy on their page as well as modify state and props for debugging purposes.


While we haven’t really nailed that “taste” thing, we were able to make imitation caesar dressing by pouring the soylent into an old Hidden Valley bottle!

If you’re interested in web development at Mozilla, or want to attend next month’s Extravaganza, subscribe to the dev-webdev@lists.mozilla.org mailing list to be notified of the next meeting, and maybe send a message introducing yourself. We’d love to meet you!

See you next month!

Beer and Tell – July 2015

Once a month, web developers from across the Mozilla Project get together to develop an encryption scheme that is resistant to bad actors yet able to be broken by legitimate government entities. While we toil away, we find time to talk about our side projects and drink, an occurrence we like to call “Beer and Tell”.

There’s a wiki page available with a list of the presenters, as well as links to their presentation materials. There’s also a recording available courtesy of Air Mozilla.

Osmose: Moseamp

Osmose (that’s me!) was up first, and shared Moseamp, an audio player. It’s built using HTML, CSS, and JavaScript, but acts as a native app thanks to the Electron framework. Moseamp can play standard audio formats, and also can load plugins to add support for extra file formats, such as Moseamp-Audio-Overload for playing PSF files and Moseamp-GME for playing NSF and SPC files. The plugins rely on libraries written in C that are compiled via Emscripten.

Peterbe: Activity

Next was Peterbe with Activity, a small webapp that displays the events relevant to a project, such as pull requests, PR comments, bug comments, and more, and displays the events in a nice timeline along with the person related to the action. It currently pulls data from Bugzilla and Github.

The project was born from the need to help track a single individual’s activities related to a project, even if they have different usernames on different services. Activity can help a project maintainer see what contributors are doing and determine if there’s anything they can do to help the contributor.

New One: MXR to DXR

New One was up next with a Firefox add-on called MXR to DXR. The add-on rewrites all links to MXR viewed in Firefox to point to the equivalent page on DXR, the successor to MXR. The add-on also provides a hotkey for switching between MXR and DXR while browsing the sites.

bwalker: Liturgiclock

Last was bwalker who shared liturgiclock, which is a webpage showing a year-long view of what religious texts that Lutherans are supposed to read throughout the year based on the date. The site uses a Node.js library that provides the data on which text belongs to which date, and the visualization itself is powered by SVG and D3.js.


We don’t actually know how to go about designing an encryption scheme, but we’re hoping to run a Kickstarter to pay for the Udacity cryptography course. We’re confident that after being certified as cryptologists we can make real progress towards our dream of swimming in pools filled with government cash.

If you’re interested in attending the next Beer and Tell, sign up for the dev-webdev@lists.mozilla.org mailing list. An email is sent out a week beforehand with connection details. You could even add yourself to the wiki and show off your side-project!

See you next month!

Extravaganza – July 2015

Once a month, web developers from across Mozilla get together to dumpster dive for hardware to add to our in-house cloud computing service. While we argue about the compute power of TI-83s, we find time to talk about the work we’ve shipped, share the libraries we’re working on, meet new folks, and talk about whatever else is on our minds. It’s the Webdev Extravaganza! The meeting is open to the public; you should stop by!

You can check out the wiki page we use to organize the meeting, or view a recording of the meeting in Air Mozilla. Or just read on for a summary!

Shipping Celebration

The shipping celebration is for anything we finished and deployed in the past month, whether it be a brand new site, an upgrade to an existing one, or even a release of a library.

DXR search package for Atom

First up was Osmose (that’s me!) with a package for Atom, a text editor made by Github. The package is called atom-dxr-search and lets you perform searches on DXR (Mozilla’s structured code search engine) from directly within your text editor. And, if you have the code tree you’re searching open as a directory in Atom, you can click on the results to open the matching file and jump directly to the line in question!

Socorro: Now on AWS!

Next was lonnen, who shared the great news that Socorro, the crash collector service that handles crash reports for Firefox and other products, has successfully migrated off of Mozilla infrastructure and on to AWS. This is the culmination of 2-3 quarters of work by the Socorro team, and will allow the team to scale and deploy much faster than before.

Along with the switch itself, the team set up several new tools to help deploy and monitor the service, including Atlas, which lets the team to audit and test infrastructure changes before they get deployed.

Air Mozilla /new page

Peterbe launched the New/Upload page on Air Mozilla, which lets users to upload existing videos or record a new video from their webcam. The page makes it much easier to submit content to Air Mozilla, and in the future the page will allow you to record screencasts as well.

DXR 2.0 Demo

ErikRose shared a link to his presentation on the new things coming in the DXR 2.0 update, as well as a link to the post-presentation discussion on what the future roadmap for DXR looks like.

Edwin, a tool for bug management

Mythmon has been working on Edwin, which is a small React app that lets you manage the list of bugs to work on for a project. It pulls data from both Bugzilla and GitHub, knows the current review status of a bug, and lets you to sort and prioritize work easily. SUMO and Input are currently using the tool to manage their work, and any projects interested in trying the tool out can contact mythmon to get their project added.

Pontoon on Heroku

I wrapped things up with the news that Pontoon, a localization service that supports in-page translation for websites, has migrated to Heroku. The jump is the result of work from both myself and mathjazz, the main developer of the site. The migration required upgrading Django from 1.4 to 1.8, replacing Git submodules with a peep-compatible requirements file, replacing jingo with django-jinja, and a slew of other changes generally around removing the last traces of playdoh from the site.

In addition, the Pontoon team is looking to help anyone interested in switching to Pontoon. If you’re not using Pontoon and want to be (and you should), let me know and I’ll be able to help start the process as well as possibly contribute patches to your site to enable in-page localization.


In the end we weren’t able to find enough computers, so we opted to install rootkits on all the company laptops that harvest unused CPU for our cloud.

If you’re interested in web development at Mozilla, or want to attend next month’s Extravaganza, subscribe to the dev-webdev@lists.mozilla.org mailing list to be notified of the next meeting, and maybe send a message introducing yourself. We’d love to meet you!

See you next month!