MDN workflow was a formless void

When I first joined Mozilla there was no workflow for MDN; in fact, there were no full-time developers for MDN! To push my first bug fix to the site, I had to negotiate a mess of mdn bugs, wiki pages, and IT bugs, strung together with email. It took a couple weeks to get a simple fix into production, and worst of all – I had to interrupt at least 3 other peoples’ regular workflows.

Let there be Scrum

I had done Scrum for a few years at SourceForge so I naturally started to work in some “scrummy” things for MDN – first I added some agile/scrum features to the Bugzilla JS addon. Then we got another couple of developers on the team and we used more agile practices in our MDN workflow. So we did user stories, standups, retrospectives, planning poker, and sprints on top of Bugzilla with Paul McLanahan’s fantastic ScrumBugs tool. We used it to run our sprints for over a year; to migrate MDN from MindTouch to Django. Sprints helped us prioritize, plan, and commit to release batches of bug-fixes, enhancements, and features at regular intervals.

Continuous Deployment

On our new Django platform, James and Jake hooked chief up to give us continuous deployment to MDN; our “regular intervals” went from weeks to hours. Our flexibility and adaptability with continuous deployment is awesome. Pushing frequently caused a series of effects on our workflow, though:

1. We naturally prioritized things by hours or days. So, during our sprints, we would
2. remove and add items between planning meetings. This had knock-on effects like
3. making it hard to see what we’re actually working on between meetings, and
4. making our sprint-based estimates totally inaccurate.

A ban came, it’s name was Kan

Around this time, I read Kanban, by David Anderson and some of Kanban and Scrum. A couple things stood out to me:

• Kanban visualizes existing workflow
• Kanban favors Evidence-Based Scheduling over estimates

The Kanban feature I most liked was lead time – i.e., the clock time between when a request is made and delivered – and cycle times – i.e., the clock time between when a cycle of work begins and ends. These clock times seem more valuable to plan and prioritize than fuzzy estimations. Because really – who doesn’t like data and numbers, right? So, of course, I wrote code – kanbugs – to help visualize our MDN workflow and calculate our lead and cycle times.

Kanbugs

Kanbugs used data from our bugzilla product and our GitHub pull requests to visualize our workflow. In general, the workflow was:

1. We select a bug and add ‘p=<estimate>’ to the whiteboard
2. We implement a bug and submit a GitHub pull request with “fix bug <id>” in the commit message
3. We review a bug and merge it to master, which automatically marks the bug ‘RESOLVED:FIXED’ in bugzilla
4. WebQA tests the bug on the MDN dev integration server
5. We release the code to production, and WebQA marks the bug “VERIFIED:FIXED” in bugzilla

By visualizing the workflow and calculating both lead and cycle times we learned a couple interesting things. Most prominently, of our 39-day lead-time, 19 days were spent in triage and 14 days were spent waiting for testing! 85% of our lead-time was happening outside the development team. This kind of data helps us more effectively improve our workflow. E.g., rather than improve our estimates, we worked with WebQA to change from mandatory to as-needed exploratory testing.

Kanbugs was good for passive, read-only visualization and calculation, but (like Scrumbu.gs before) it wasn’t made for active management. It lacks:

• Quick visual scanning for action items – who’s blocked? who’s too busy? what should I work on next?
• Work-in-Progress limits – Kanban enforces small batch sizes by setting limits on how many things can be in the board at a time

Kanbanery

Kanbanery is good for these features. So, a couple months ago, we started an MDN board on Kanbanery. Kanbanery’s visualization is great. On a typical morning, I check emails and then

1. quickly scan the board for any cards with green check-marks that I can pull to the next step of development.
2. quickly scan the work-in-progress limits to see if I can help un-block cards that are piling up at a certain step
3. quickly scan the board for my cards to resume my main dev task(s)

Of course, now that we’ve used Kanbanery for a couple months, we’ve hit drawbacks:

• Yet Another Tracking Tool – we already have to keep up with email, bugzilla, and github. Scrumbugs had the same drawback, but Scrumbugs doesn’t micro-track through the sprint, and it automatically marks bugs fixed using bugzilla data.
• How to incorporate contributors – we don’t want to make contributors use Yet Another Tracking Tool, so we don’t include them in the board. That means the board doesn’t show contributors’ work, which can be considerable.
• General process churn – we went from no process through multiple forms of agile and scrum practices to Kanban in the space of about 18 months. I really like to continuously improve dev practices, but even I’m getting tired of changing things.

What’s next – maybe Kanbuggery?

Rather than change things up again, what I’d really like to do is automate Kanbanery – to update the board based on bugzilla and github actions, like we had done with Kanbugs. Then Kanbanery could give us its great visual features without needing active care and attention. Kanbanery has what looks like a fully-featured API, but I haven’t had time to explore it. Has anyone else?

How are other Mozilla, open-source, or website teams using Kanban for development?