Beer and Tell July 2014

Michael Kelly

0

Once a month, web developers across the Mozilla community get together to share what side projects or cool stuff we’ve been working on in our spare time. This monthly tribute is known as “Beer and Tell”.

There’s a wiki page listing the presenters and links to what they’re showing off and useful side information. There’s also a recording on Air Mozilla of the meeting.

Pomax: RGBAnalyse and nrAPI

This month Pomax had two projects to show. The first, RGBAnalyse, is a JavaScript library that generates histographical data about the colors in an image. Originally created so he could sort ink colors by hue, the library not only generates the data, but also generates images (available as data-uris) of histograms using that data.

The second project Pomax shared was nrAPI, a Node.js-based REST API for a website for learning Japanese: nihongoresources.com. The API lets you search for basic dictionary info, data on specific Kanji, sound effects, and Japanese names. Search input is accepted in English, Romaji, Hiragana, Katakana, or Kanji.

HTML result from nrAPI search for "tiger".

HTML result from nrAPI search for “tiger”.

Bill Walker: Photo Mosaic via CSS Multi-Column

Next, bwalker shared his personal birding photo site, and talked about a new photo layout he’s been playing with that uses a multi-column layout via CSS. The result is an attractive grid of photos of various sizes without awkward gaps, that can also be made responsive without the use of JavaScript. bwalker also shared the blog post that he learned the technique from.

Dean Johnson: MusicDownloader

deanj shared MusicDownloader, a Python-based program for downloading music from SoundCloud, Youtube, Rdio, Pandora, and HypeScript. The secret sauce is in the submodules, which implement the service-specific download code.

Chris Lonnen: Alonzo

Lastly, lonnen shared alonzo, a Scheme interpreter written in Haskell as part of a mad attempt to learn both languages at the same time. It uses Parsec to implement parsing, and so far implements binary numeric operations, basic conditionals, and even rudimentary error checking. The development roughly follows along “Write Yourself a Scheme in 48 Hours” by Johnathan Tang.

Sample Runs of alonzo


Thanks to all of our presenters for sharing! If you’re interested in attending or presenting at the next Beer and Tell, subscribe to the dev-webdev mailing list! The wiki page and connection info is shared a few days before each meeting.

See you next month!

Webdev Extravaganza July 2014 Notes

Michael Kelly

Once a month, web developers across the Mozilla Project get together to talk about the things we’ve shipped, discuss the libraries we’re working on, introduce any new faces, and argue about pointless things. We call it the Webdev Extravaganza! It’s open to the public; you should come next month!

There’s a wiki page that’s mostly useful for pre-meeting connection info and agenda-building, a recording of the meeting, and an etherpad which is a mix of semi-useful info and avant-garde commentary. This month, I’m trying out writing a post-meeting blog post that serves as a more serious log of what was shared during the meeting.

Shipping Celebration

The shipping celebration is for sharing anything that we have shipped in the past month, from incremental updates to our sites to brand new sites to library updates.

Mobile Partners

mobilepartners.mozilla.org is a Mezzanine-based site that allows phone manufacturers and operators to learn about Firefox OS and sign up as a partner. This month we shipped an update that, among other things,  tightens up the site’s Salesforce integration, replaces HTML-based agreements with PDF-based ones displayed via PDF.js, and moves the site off of old Mozilla Labs hardware.

Input

input.mozilla.org got two major features shipped:

  1. All non-English feedback for Firefox OS is now being automatically sent to humans for translation. Having the feedback translated allows processes that work on English only, like Input’s sentiment analysis, to run on feedback from non-English users.
  2. An improved GET API for pulling data from Input, useful primarily for creating interesting dashboards without having to build them into Input itself.

Open Source Citizenship

Peep has an IRC channel

Peep is a wrapper around pip that vets packages downloaded from PyPI via a hash in your requirements file. There’s now an IRC channel for discussing peep development: #peep on irc.mozilla.org.

Spiderflunky

Spiderflunky is a static analysis tool for JavaScript that was modified from a tool for inspecting packaged apps called perfalator. DXR is planning to use it, and there’s interest around the project, so it has it’s own repository now.

New Hires / Interns / Volunteers

This month we have several new interns (including a bunch from the Open Source Lab at OSU):

Name IRC Nick Project
Trevor Bramwell bramwelt crash-stats.mozilla.com
Ian Kronquist muricula input.mozilla.org
Dean Johnson deanj support.mozilla.org
Marcell VazquezChanlatte marcell dxr.mozilla.org
Christian Weiss cweiss Web Components / Brick

Bikeshed / Roundtable

In the interests of time and comprehensibility, open discussions will be ignored.

What does Webdev do?

As part of an ongoing effort to improve webdev, there’s an etherpad for listing the things that webdev does, along with who owns them, what goals they contribute to, and how they can be improved. You are encouraged to take a look and add info or opinions by the end of the week.

Interesting Python Stuff

Erik Rose shared a backport of concurrent.futures from Python 3.2 and a feature from setuptools called entry points that is useful for, among other things, supporting a plugin architecture in your library.


Did you find this post useful? Let us know and we might do it again next month!

Until then, STAY INCLUSIVE, KIDS

New Firefox Nightly firstrun page and community growth

Theo Chevalier

1

This summer I’m doing an internship at Mozilla, and so far that’s the most exciting time of my life. I work in the Localization Team in Mountain View, and one of my goals is to increase the number of users on localized Firefox Nightly builds.
Several projects are ramped up to reach this goal, and one of them is the Nightly firstrun page. The firstrun page is the web page you see the first time you run Nightly (makes sense, right?), but also each time we release a new Firefox.

Why do Firefox Nightly builds matter?

Firefox Nightly is the earliest Firefox branch, on which all Firefox developers push their work. This build may be unstable, (but I’ve been using it as my main browser for more than 3 years, so far not that many issues). Most of the users are technical users, and they would be really valuable to the Mozilla mission if we could find an easy way to get them more involved in the project. Here comes the firstrun!

Nightly firstrun in English

Why does the firstrun page matters?

This page is one of the first things we show to our technical users. The page has been simplified on purpose, to give them a clear way to get involved. 3 core areas are displayed: coding, QA, and localization. For instance, they can find a way to provide feedback to the localization team of their language, this is really helpful for localizers to get early feedback. They can even join the team! We really want users to get more involved with their local community. That’s why we don’t just get the page translated, we localized it by adding a customizable block at the bottom of the page. This allows Mozilla communities to create and publish their own content, (for instance to promote local events, IRC channel…) so that users can meet them and get involved more quickly and easily. Fun fact, I was myself initially using Nightly and contributing to Firefox code before even knowing a French community existed!
In order to always get relevant content, localizers can update the block as part of the general Web Localization process, so it can be done quickly and regularly, without having to ask anyone.

The page has been live for a few days now, and you’ll see it if you create a new profile on Firefox Nightly, or next time Firefox team bumps up the version number. You can take a look at the per-locale customization on the French page below for instance. The page is already localized in a few locales, also we are planning to add more locales amongst Firefox Nightly locales really soon. The next step for the page will be to analyze the traffic and determine what we can improve.

Nightly firstrun in French

You too, get more involved!

If you want to contribute to these areas, you can download Firefox Nightly in your language. Right now this is the only link we have to get people to download localized Firefox Nightly builds, but we plan to get them more exposed really soon by adding Firefox Nightly to the new design of the channel page.

Personal notes

I learned a lot during this project, this was my first page for mozilla.org, and this was a fun challenge. Our Creative Team created the new design, I did the HTML/CSS code — and of course, made sure the page was l10n friendly — and the Mozilla.org dev team reviewed my work and gave me useful tips. There were a lot of things to take into account: performance, accessibility, localization, metrics, responsive design… but it was really worth it.
Also, a huge thank you to Pascal Chevrel and Delphine Lebédel my mentors, and to all the Mozillians for helping me learning new things!

Site launch: Affiliates 2.0!

Justin Crawford

We just launched a redesigned website for Firefox Affiliates — check it out!

New Affiliates web site!

The Affiliates website is a portal for almost 100,000 Mozilla supporters. Affiliates believe in Mozilla’s cause and want to show their support by placing an affiliate link on their website. Here’s mine:

The original Affiliates website launched in 2011. Its branding recalls an earlier era in Mozilla style. I will miss those fuzzy monsters, but (for now at least) I know I can still find them on Mozilla.org’s 404 page. And seriously, after three years, the site needed more than a brand refresh. As part of this launch we…

Image from the original front page.

The Affiliates 1.0 front page included fuzzy monsters.

  • Upgraded to the latest version of Playdoh
  • Removed lots of old code and subsystems no longer in use
  • Reconsidered and redesigned the entire user experience
  • Updated the pages to a beautiful take on Mozilla’s Sandstone theme
  • Added an affiliate leaderboard
  • Added user profiles and a personalized dashboard
  • Added space for localized news and announcements
  • Much more!

If you’re already a Firefox Affiliate, we* hope you enjoy these improvements. And if you’re not, why not sign up now? Being an affiliate is an easy way to support the efforts of a non-profit that strives every day to build the web the world needs.

We’d like to hear from you. Please file a bug if you find one or reach out in the #affiliates channel on irc.mozilla.org to say “hi”. And keep your eye out for more new features — there’s another batch of Affiliates Awesome just around the corner!

* The Affiliates 2.0 team included dozens of people spanning the globe — too many to list here. This site redesign was a collective effort of Mozilla’s amazing affiliates, volunteers, localizers, designers, copywriters, engineers, quality assurers, contractors,  and many others. Thank you all.

Celebrating the Web We Want

Benjamin Sternthal

197

promo-1

The latest version of Firefox is a huge milestone for Mozilla and represents the combined efforts of hundreds of people from around the world. To celebrate this launch we revisited Glow, a real-time visualization of Firefox downloads originally used during the launch of Firefox 4 in 2011.

This time we wanted to do more than just show downloads. We wanted to give people a chance to express what they want the Web to be. To this end, users visiting https://webwewant.mozilla.org/ have the option to share with Mozilla and the world their hopes for the Web. These “shares” are displayed on the map and we display information about which issues are most important in each continent and country.

promo-2promo-3

The dots on the map represents people: either someone downloading Firefox or sharing the Web they want. For me, watching the map is both humbling and inspiring. When people use Firefox— a free and open source browser made by an amazing community— and choose to share with the world what matters to them, it shows a global commitment to the future of the Web.

Technical Details

How do we process millions of interactions and display them?

The site processes request logs for Firefox downloads, auto-updates, and users who share via the website. We perform geolocation lookups on the data and aggregate the information for consumption and display by the Web application.

We have a particular love of the Simpsons on our team and elements of our project are named appropriately:

  • Mr. Burns – The overall codebase.
  • Smithers – The “servant” scripts who provide data to Mr. Burns.

Mr. Burns relies on three main scripts to display data:

  • Bart – Parses log files looking for Firefox downloads or auto-updates.
  • Lisa – Geolocates parsed data and aggregates into buckets.
  • Milhouse – Packages data into JSON for consumption by the front-end.

The flow of data can be seen in this diagram:

glow-data

Data is stored in Redis and the JSON files are delivered to the front end application via CDN. The front end app renders the individual “glows” and the stats using D3.js.

To learn more visit our Wiki and explore the code on Github.

Special Thanks

This was a large project with many participants, all of whom deserve thanks. I would like to thank a few people who worked especially hard:

Eric Petitt – Project Sponsor

Paul McLanahan – Senior Developer

Steven Garrity – Senior Developer

Ali Almossawi – Metrics Engineer

John Slater – Creative Director

Sean Martell – Art Director

Matej Novak – Copy Director

The Localization Team & Contributors

And you! Without you the world would have no Glow.

Do you want a free jQuery Foundation membership?

groovecoder

jQuery Foundation Logo

Mozilla is renewing our jQuery Foundation corporate membership for 2014, and giving 2 individual memberships to web developers in the Mozilla community!

If you want us to consider you for a membership:

1. Join the ‘jquery’ group on Mozillians
2. Edit your Mozillians bio to include your most impressive jQuery code or activities

By June 2014, Mozilla Developer Relations and Webdev teams will select 2 winners based on the most outstanding contributions to the Open Web using jQuery.

Keep rocking the free web!

Enabling Communities: Iterating on the Get Involved Page

jbertsch

2

The Get Involved page on mozilla.org is one of the primary ways future community members reach out to join Mozilla and work with us to build a better Internet.

In March 2013, we turned the Get Involved page into an animated slideshow celebrating Mozilla’s 15th anniversary.

15th Anniversary slide show on the Get Involved Page

15th Anniversary slide show on the Get Involved Page

As the end of Mozilla’s 15th anniversary year approached, we wanted begin iterating on an improved user sign-up flow and telling a different part of the Mozilla story.  We also wanted to make sure our storytelling supports open web standards, so we included a sequence of HTML5 videos showing just a few of the amazing members of our global community.

You can see the updated page now in English, French, Italian, and German, with more locales to be added very soon.

What’s next for the Get Involved page?

The www.mozilla.org team is working closely with the Community Building and Community Engagement teams to build a new version of the Get Involved page to better support Mozilla’s  “Enable Communites that Have Impact”  goal.

We will collaborate with the Community Builder’s Research group to better understand how and why prospective new community members find Mozilla.  We’ll also coordinate closely with the Pathways and Systems & Data groups to ensure we are doing everything we can to help potential volunteers find the opportunity they are looking for to work with Mozilla.

We need your help!

We are still in the “gather data/understand the problem” phase of this next iteration.  If you see other websites that are doing a great job recruiting volunteers or have ideas for things that the Get Involved page could do to better connect people with Mozilla (ex:  make it easier for people to find Mozilla events close to home), please let us know!

You can contribute your ideas by either:

1.  adding them to this etherpad

2.  creating dependent bugs to this tracking bug

Expect to see the next updates to the Get Involved page in Q3 2014!

Building A JavaScript Library With Tests, Mocks, and CI

kumar303

I talked recently at DjangoCon about how to build better Python packages. As engineers we all use each other’s open source libraries everyday so it’s nice when 1) they work, 2) they’re well documented, and 3) they’re well tested.

For a recent project I set out to build a JavaScript library for other developers to use in their web applications. It had been a while since I made a browser library in JavaScript so I took it as an opportunity to see if I could apply some Python packaging concepts to JavaScript.

This is a brief summary of some tools and patterns I settled on. There are plenty of alternatives so if you have suggestions for improvement, please comment!

Project Tasks

With any project it’s inevitable that you’ll begin writing scripts for maintenance. I decided to go with Grunt because it keeps me in the realm of JavaScript (even though the developer needs NodeJS installed). Plus there are lots of contributed Grunt tasks to choose from. For example, to minify my library at lib/myscript.js all I had to do was install uglify-js, grunt-contrib-uglify, and add a Gruntfile.js with this:

Now I can type this to minify the code:

grunt uglify

Running Unit Tests

All libraries (even JavaScript ones!) need automated tests otherwise you and future collaborators will find the library really hard to maintain. If you try to make a change to code without tests you have to think about every possible side effect or you have to manually test all side effects. Both of these things seem easy when you only have one or two functions but they become exponentially harder as your library grows.

I decided to try Karma to run tests in a real web browser, mocha for my test cases, and chai for assertions. There are a few alternatives but Karma seemed like the best choice for a runner. (I found out about Yeti after I settled on Karma; I haven’t tried it.)

Karma lets you run tests from your console against one or more web browsers in parallel. It’s exactly what I wish existed back when I wrote jstestnet! Karma is really fast and seems to work pretty well. Speed is essential in testing.

To get started, you can type karma init karma.conf.js to create a config file and you can hook it up to Grunt with grunt-karma in a Gruntfile like this:

This fit well with my project since I was already using Grunt. I can now type:

grunt test

Karma will open all target web browsers (per config file), run my test suite, report results, and shut down the web browsers.

The Karma output is pretty minimal. It looks something like this in a shell:

Screen Shot 2014-03-19 at 2.56.49 PM

For development I also added this command:

With that I can type:

grunt karma:dev

This opens all target web browsers, runs the test suite, and keeps the web browsers running. As I edit files, it re-runs the tests. This seems to work okay but occasionally I’ve seen some timeout errors that go away if I restart the browsers. I’m still looking into that.

Writing Tests

Mocha is a testing library that works in NodeJS and also in web browsers. It uses the BDD (behavioral driven development) style of specifying an object or function and declaring how it should behave. Here is an example from my library that covers some error handling:

As you can see it’s pretty easy to read the code and see what is being tested. Both Mocha and Jasmine have Karma adapters and there are probably other adapters too if you want to use another test library.

Testing With Mock Objects

A common pattern in testing is to mock out objects that are used by your system but that you don’t need to test. Sinon is designed exactly for this and especially for testing in the web browser. My library has a thin API layer around XMLHttpRequest but I wanted to mock that out while testing the API layer. Here is an example of making sure it gets an error callback for 500 responses:

This is nice because I can run my tests without the code touching a real API server. I’m using Sinon’s Fake Server here.

It’s easy to go overboard with mocks once you discover their power. My words of caution is that anytime you mock out an object you are deciding not to test something. Make sure that’s the right decision. Make sure it gives you real benefits like a speedup or something. It’s usually a good idea to mock out HTTP connections since otherwise your tests depend on the Internet.

Continuous Integration

Testing is most effective when you run your tests after every code commit. There is a free service for open source projects called TravisCI which supports running browser tests with Karma just fine. You can run all your tests in a headless browser like PhantomJS with a task like this:

Add the grunt command to a .travis.yml file to hook it up:

However, my specific library will benefit most from running its tests in a real Firefox and TravisCI supports web browsers just fine so, hey, why not. All I had to do was add this to my yaml file:

Firefox is pre-installed on TravisCI so I didn’t even need to declare it.

Checking For Syntax Errors

JavaScript is a quirky language (to put it nicely) so I always make sure to run the code through something like JS Hint to catch undeclared variables and other problems. You can add syntax checking easily to grunt like this using grunt-contrib-jshint:

This lets me type grunt jshint to check for syntax errors in all lib and test files as well as in my Gruntfile.js. I use a .jshintrc file to set common options that I like.

My continuous integration script actually runs JS Hint before running unit tests. I chain all these together like this at the bottom of my Gruntfile:

grunt.registerTask('test', ['jshint', 'karma:run']);

Now when I run grunt test the jshint and karma:run tasks are executed.

Documentation

All packages need documentation! I like to start with realistic scenarios to illustrate usage of the library. After that I make sure to fully document all public APIs. I tend to just put docs in a Github README and then switch to Sphinx and Read The Docs later when the docs are too big for a single README. Sphinx doesn’t support JavaScript as well as Python but it’s still probably the best tool out there for managing interlinked docs.

Packaging

To make a library useful to others you may wish to package it up. To be honest I haven’t done this to my library yet but I was considering bower and/or NodeJS packaging.

That’s It

Since I left out a lot of details, you may want to check the actual code for working examples.

A new update experience for Australis: our process and design principles

Holly Habstritt Gaal

1

We’re excited to release a new onboarding experience for users updating to the new Firefox (Australis). We’re not just introducing a new design in Firefox 29, but a new way for the web and chrome to interact with each other in our onboarding experience.  This will allow us to show, not just tell the users what is new in Firefox and educate them about the browser. With this new interaction we avoid relying on passive viewing of a web page and create a memorable experience that is immersed in the browser. To learn more, see the following blog posts that describe our design principles and process.

5 questions to ask during your design process

Learn how our initial assumptions about educating users became a project dedicated to creating an Update experience for Firefox 29 in 5 questions to ask during your design process.

Introducing the Update Experience for Australis

Learn about the key design principles that stemmed from our research and testing in Introducing the Update Experience for Australis.

 

Onboarding Tour UI

Onboarding Tour UI

Cross-team collaboration at Mozilla has been key to creating this experience.  The collaboration spans across teams such as release engineering, web development, marketing, visual design, UX, user advocacy & support, metrics, and others. Thanks to everyone involved for their hard work and stay tuned as we continue to iterate and improve this experience in preparation for the Firefox 29 general release.

We love to discuss all things onboarding. If you have any questions, please reach out to Holly Habstritt Gaal and Zhenshuo Fang.

DXR Gets A Huge UI Refresh

Erik Rose

1

After months of hard work by talented Mozillians, both paid and volunteer, DXR’s UI overhaul branch has hit production! With more efficient workflows, support for multiple trees, and improvements in discoverability, it makes searching Mozilla’s codebases more fun and takes a big step toward the retirement of legacy tools.

What’s new?

Improved flow. The old design forced you into a choice upfront about whether you’d be browsing or searching. That splash screen is gone, replaced with useful information: the top level of the source tree.

The front page of DXR, showing both the search pane and a top-level listing of the folders in mozilla-central

If you want to browse, browse; search, search. In addition, multi-tree support is polished and proven, and new trees are coming soon. Finally, breadcrumbs are integrated more smoothly into the workflow; it will soon be a one-click matter to limit search to the folder you’re browsing.

Filters upfront. We now expose and document all 27 search filters in a ubiquitous drop-down menu.

DXR's 27 filters, arrayed in the Filters menu

Previously, we showed only about 6, and even those were available only via the Advanced Search panel, which didn’t appear until you had already entered a search and hit Return—search-as-you-type didn’t cut it. Take a look: DXR knows some tricks you weren’t aware of.

Real parsing. There’s an honest-to-goodness query parser now! You can express quotation marks without doing backflips, and the quoting behavior of regexes is unified with that of the other filters. Any filter’s argument can be quoted with ether single and double quotes, and, in case you need both at once, they can be backslash-escaped. For example…

  • A phrase with a space: "big, small"
  • Quotes in a plain text search, taken as literals since they’re not leading: id="whatShouldIDoContent"
  • Double quotes inside single quotes, as a filter argument: regexp:'"wh(at|y)'
  • Backslash escaping: "I don't \"believe\" in fairies."

Furthermore, we have plans to simplify the selection of filters. You’ve said you don’t care, most of the time, what kind of identifier you’re looking for; identifier names are pretty unique. Thus, we plan generic “id” and “ref” filters to save your brain cycles. We’ll also reduce redundancy and make some things shorter and more memorable. See our sketch on the wiki, and don’t hesitate to scribble your feedback on it.

Better URL handling. The URL is updated in place as you search, so all your searches are bookmarkable and shareable, whether you hit Return or not.

Even more. The case-sensitive checkbox has an accesskey. The search field no longer autofocuses, making it easier to use arrow-key scrolling or type-to-select. Table rows present easier click targets. Infinite scrolling is more anticipatory. The JS has been completely rewritten. And everything looks prettier, to boot.

DXR viewing a syntax-highlit file, with a context menu open offering to jump to a definition of AtkAttribute

Thanks to everyone who has contributed their feedback and expertise to this release, not only to the UI but also to the back-end improvements that went into production while it was cooking. Special recognition goes to Schalk Neethling for his front-end magic; Nick Cameron, who has been making things better all over the stack; and James Abbatiello, who keeps adding new filters and chasing down analysis corner cases. It’s a fun project to hack on, with something for everybody. Join us!

More is always to come. Known issues are here. File anything else you see.

Happy searching!