More details about how to use the “tracking-firefox#” Bugzilla flag

In the new channel triage meetings we’ve been noticing some misuse of the new tracking flag. Hopefully this post will help clear up when and how to use it.

What is tracking-firefox# for?

Bugs marked tracking-firefox# are bugs that must be resolved one way or another before a particular release ships. Release drivers will track and shepherd the bug until it is determined the bug no longer impacts the release.

Tell us why you are nominating for tracking-firefox#

There should be a clear argument as to why release drivers should care.

Good examples include:

  • The bug fixes a (new or existing) top crash
  • The bug causes a top crash
  • The bug has the potential to impact web compatibility
  • The bug is a regression that has not shipped to the public yet
  • The bug has add-on compatibility implications release drivers should be aware of
  • The bug introduced a new feature which may need to be backed out if feedback is negative
  • The bug is requesting a feature backout on mozilla-aurora or mozilla-beta

Bugs without justification will be sent back for more information. Save everyone time by giving the proper information up front.

“It’s a bad bug” is NOT justification. The first question we’ll ask is how we know it’s bad (spike on crash-stats, large issue on SUMO reports, breaking a top site, etc). The second will be why it needs to be fixed now rather than land on mozilla-central and be uplifted in a future release.

Bugs with tracking-firefox# have actionable next steps

Bugs marked tracking-firefox# should have actionable next steps. Often, the next thing to do is to gather more information (from automated systems like crash-stats or the bug reporters themselves). If a bug has tracking-firefox# set it means release drivers will help shepherd those next steps until it becomes clear the bug will not impact a release (either via a fix or new information). At that point, release drivers will no longer track the bug.

tracking-firefox# should not be used as a general way to prioritize fixes

The individual engineering teams and module owners should have their own way of tracking what needs to be fixed and in what order. Setting tracking-firefox# will likely put a bug at the top of that prioritization but it should not take the place of a general priority system.

We are purposely not dictating the way to track prioritization in Bugzilla as each team has a different method (some use target milestone, some use whiteboard entries, etc).

Bugs denied for tracking-firefox# are still important

If release drivers decide not to track a bug for a certain version of Firefox it does not mean the bug is unimportant. It merely means–based on the information we have now–release drivers do not feel the bug would prevent us from shipping a release.

If new information comes to light, you need help getting more data before you can make the case for release drivers to track, or you disagree with our assessment feel free to renominate again with additional justification.

tracking-firefox# is not the same as blocking in the old release process

Initially, the old blocking flag was intended to track only those bugs preventing us from shipping a release to the public. It was quickly recognized that bugs with the blocking flag were higher priority, got investigated and resolved more quickly, and never feel through the cracks. Suddenly, everyone started nominating pet bugs, bugs that seemed bad upon initial reading but could not be reproduced, tracking bugs for non-critical peripheral work that had to happen around the same time as a release, bugs to “keep an eye on” with no actionable work, etc.

With everyone nominating bugs they were watching the blocking list got so big and unruly we eventually had to segment it.  We came up with a concept of “hardblockers” (bugs that really would block the release) and “softblockers” (everything else people thought was important). This let people have the comfort of marking something as blocking while still allowing release drivers to focus on the minimal set of bugs to be fixed.

From a tools perspective we flattened Bugzilla’s target milestone, priority, keyword, and flag system into one flag with whiteboard annotations. Clearly not ideal, and we have no intention of repeating the outcome with the new development process.

tracking-firefox# will NOT become a large list of bugs we continually roll forward. We intend to treat it as the minimal set of bugs needing daily scrutiny from release drivers to make sure a specific release is up to Mozilla’s standards.