{"id":187,"date":"2020-05-08T15:16:08","date_gmt":"2020-05-08T15:16:08","guid":{"rendered":"https:\/\/blog.mozilla.org\/data\/?p=187"},"modified":"2020-05-08T15:16:08","modified_gmt":"2020-05-08T15:16:08","slug":"this-week-in-glean-mozregression-telemetry-part-2","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/data\/2020\/05\/08\/this-week-in-glean-mozregression-telemetry-part-2\/","title":{"rendered":"This Week in Glean: mozregression telemetry (part 2)"},"content":{"rendered":"

(\u201cThis Week in Glean\u201d is a series of blog posts that the Glean Team at Mozilla is using to try to communicate better about our work. They could be release notes, documentation, hopes, dreams, or whatever: so long as it is inspired by Glean. You can find an index of all TWiG posts online.<\/a>)<\/em><\/p>\n

This is a special guest post by non-Glean-team member William Lachance!<\/em><\/p>\n

This is a continuation of an exploration of adding Glean-based telemetry to a python application, in this case mozregression<\/a>, a tool for automatically finding the source of Firefox regressions (breakage).<\/p>\n

When we left off last time<\/a>, we had written some test scripts and verified that the data was visible in the debug viewer.<\/p>\n

Adding Telemetry to mozregression itself<\/h2>\n

In many ways, this is pretty similar to what I did inside the sample application: the only significant difference is that these are shipped inside a Python application that is meant to be be installable via pip<\/a>. This means we need to specify the pings.yaml<\/code> and metrics.yaml<\/code> (located inside the mozregression<\/code> subirectory) as package data inside setup.py<\/code>:<\/p>\n

\n
\n
setup<\/span>(<\/span>\r\n    name<\/span>=<\/span>\"mozregression\"<\/span>,<\/span>\r\n    ...<\/span>\r\n    package_data<\/span>=<\/span>{<\/span>\"mozregression\"<\/span>:<\/span> [<\/span>\"*.yaml\"<\/span>]},<\/span>\r\n    ...<\/span>\r\n)<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n
There were also a number of Glean SDK enhancements which we determined were necessary. Most notably, Michael Droettboom added 32-bit Windows wheels to the Glean SDK, which we need to make building the mozregression GUI<\/a> on Windows possible. In addition, some minor changes needed to be made to Glean\u2019s behaviour for it to work correctly with a command-line tool like mozregression \u2014 for example, Glean used to assume that Telemetry would always be disabled via a GUI action so that it would send a deletion ping, but this would obviously not work in an application like mozregression where there is only a configuration file \u2014 so for this case, Glean needed to be modified to check if it had been disabled between<\/em> runs.<\/p>\n
Many thanks to Mike (and others on the Glean team) for so patiently listening to my concerns and modifying Glean accordingly.<\/p>\n
Getting Data Review<\/h2>\n
At Mozilla, we don\u2019t just allow random engineers like myself to start collecting data in a product that we ship (even a semi-internal like mozregression). We have a process<\/a>, overseen by Data Stewards to make sure the information we gather is actually answering important questions and doesn\u2019t unnecessarily collect personally identifiable information (e.g. email addresses).<\/p>\n
You can see the specifics of how this worked out in the case of mozregression in bug 1581647<\/a>.<\/p>\n
Documentation<\/h2>\n
Glean has some fantastic utilities for generating markdown-based documentation on what information is being collected, which I have made available on GitHub:<\/p>\n
https:\/\/github.com\/mozilla\/mozregression\/blob\/master\/docs\/glean\/metrics.md<\/a><\/p>\n
The generation of this documentation is hooked up to mozregression\u2019s continuous integration<\/a>, so we can sure it\u2019s up to date.<\/p>\n
I also added a quick note<\/a> to mozregression\u2019s web site describing the feature, along with (very importantly) instructions on how to turn it off.<\/p>\n
Enabling Data Ingestion<\/h2>\n
Once a Glean-based project has passed data review, getting our infrastructure to ingest it is pretty straightforward. Normally we would suggest just filing a bug<\/a> and let us (the data team) handle the details, but since I\u2019m on<\/em> that team, I\u2019m going to go a (little bit) of detail into how the sausage is made.<\/p>\n
Behind the scenes, we have a collection of ETL (extract-transform-load) scripts in the probe-scraper repository<\/a> which are responsible for parsing the ping and probe metadata files that I added to mozregression in the step above and then automatically creating BigQuery tables and updating our ingestion machinery to insert data passed to us there.<\/p>\n
There\u2019s quite a bit of complicated machinery being the scenes to make this all work, but since it\u2019s already in place, adding a new thing like this is relatively simple. The changeset I submitted as part of a pull request<\/a> to probe-scraper was all of 9 lines long:<\/p>\n
\n
\n
diff --git a\/repositories.yaml b\/repositories.yaml<\/span>\r\nindex dffcccf..6212e55 100644<\/span>\r\n--- a\/repositories.yaml<\/span>\r\n+++ b\/repositories.yaml<\/span>\r\n@@ -239,3 +239,12 @@ firefox-android-release:<\/span>\r\n     - org.mozilla.components:browser-engine-gecko-beta\r\n     - org.mozilla.appservices:logins\r\n     - org.mozilla.components:support-migration\r\n+mozregression:<\/span>\r\n+  app_id: org-mozilla-mozregression<\/span>\r\n+  notification_emails:<\/span>\r\n+    - wlachance@mozilla.com<\/span>\r\n+  url: 'https:\/\/github.com\/mozilla\/mozregression'<\/span>\r\n+  metrics_files:<\/span>\r\n+    - 'mozregression\/metrics.yaml'<\/span>\r\n+  ping_files:<\/span>\r\n+    - 'mozregression\/pings.yaml'<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n
A Pretty Graph<\/h2>\n
With the probe scraper change merged and deployed, we can now start querying! A number of tables are automatically created according to the schema outlined above: notably \u201clive\u201d and \u201cstable\u201d tables corresponding to the usage ping. Using sql.telemetry.mozilla.org<\/a> we can start exploring what\u2019s out there. Here\u2019s a quick query I wrote up:<\/p>\n
\n
\n
SELECT<\/span> DATE<\/span>(<\/span>submission_timestamp<\/span>)<\/span> AS<\/span> date<\/span>,<\/span>\r\n       metrics<\/span>.<\/span>string<\/span>.<\/span>usage_variant<\/span> AS<\/span> variant<\/span>,<\/span>\r\n       count<\/span>(<\/span>*<\/span>),<\/span>\r\nFROM<\/span> `<\/span>moz<\/span>-<\/span>fx<\/span>-<\/span>data<\/span>-<\/span>shared<\/span>-<\/span>prod<\/span>`<\/span>.<\/span>org_mozilla_mozregression_stable<\/span>.<\/span>usage_v1<\/span>\r\nWHERE<\/span> DATE<\/span>(<\/span>submission_timestamp<\/span>)<\/span> >=<\/span> '2020-04-14'<\/span>\r\n  AND<\/span> client_info<\/span>.<\/span>app_display_version<\/span> NOT<\/span> LIKE<\/span> '%.dev%'<\/span>\r\nGROUP<\/span> BY<\/span> date<\/span>,<\/span> variant<\/span>;<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n
Generating a chart like this:<\/p>\n
\"\"<\/div>\n
This chart represents the absolute volume of mozregression usage since April 14th 2020 (around the time when we first released a version of mozregression with Glean telemetry), grouped by mozregression \u201cvariant\u201d (GUI, console, and mach) and date – you can see that (unsurprisingly?) the GUI has the highest usage.<\/p>\n
Next Steps<\/h2>\n
We\u2019re not done yet! Next time, we\u2019ll look into making a public-facing dashboard demonstrating these results and making an aggregated version of the mozregression telemetry data publicly accessible to researchers and the general public. If we\u2019re lucky, there might even be a bit of data science<\/em>. Stay tuned!<\/p>\n
(( This is a syndicated copy of the original post<\/a>. ))<\/p>\n","protected":false},"excerpt":{"rendered":"
(\u201cThis Week in Glean\u201d is a series of blog posts that the Glean Team at Mozilla is using to try to communicate better about our work. They could be release … Read more<\/a><\/p>\n","protected":false},"author":1528,"featured_media":168,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[448297],"tags":[448297,288565],"coauthors":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/posts\/187"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/users\/1528"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/comments?post=187"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/posts\/187\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/media\/168"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/media?parent=187"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/categories?post=187"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/tags?post=187"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/coauthors?post=187"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}