For those that don’t know, the Graph Server is used to display performance test data of Firefox builds reported by Talos.
Our work initially started as performance improvements and some new features, but the more we worked with the old architecture, it became quite apparent it would not scale (performance and feature-wise).
If it’s not clear already, one of main issues was with the database schema; it needed to be normalized.
Here’s the old, non-normalized schema:
And here’s the new, normalized schema after the team was locked in a room for an afternoon:
Much cleaner, no duplicated data, easy to understand the various machines, branches and tests that are used for displaying test data. No need look at entire tables to find basic information such as test names.
With this new schema in place, it also required a re-write of our server-side scripts we use to fetch test information for the front end graphing component. Since Mozilla is as open as possible, instead of just changing what was needed, I decided to implement a JSON API that would allow anyone to easily retrieve test data.
Lastly, our Talos <--> Graph Server communication needed to be re-written. Lars rewrote the collector script that accepts values from Talos and Alice rewrote the pieces of Talos that send data to the Graph Server.
After all that work, we now have a working stage server (Firefox 3.1 or higher required due to native JSON requirement) with our new code. We have a bit more testing and some performance benchmarking to do before it goes live, but we’re happy that all the pieces are working.
Want to know more? We have a wiki page with more information at https://wiki.mozilla.org/Perfomatic#Rearchitecture.