{"id":137,"date":"2020-03-25T12:00:13","date_gmt":"2020-03-25T12:00:13","guid":{"rendered":"https:\/\/blog.mozilla.org\/data\/?p=137"},"modified":"2020-03-25T15:13:34","modified_gmt":"2020-03-25T15:13:34","slug":"this-week-in-glean-comparing-apples-to-oranges","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/data\/2020\/03\/25\/this-week-in-glean-comparing-apples-to-oranges\/","title":{"rendered":"This Week in Glean: Comparing Apples to Oranges"},"content":{"rendered":"

(\u201cThis Week in Glean\u201d is a series of blog posts that the Glean Team at Mozilla is using to try to communicate better about our work. They could be release notes, documentation, hopes, dreams, or whatever: so long as it is inspired by Glean. You can find<\/i> an index of all TWiG posts online.<\/i><\/a>)<\/i><\/p>\n

One of my favorite parts of working on the Telemetry Team is the wide variety of projects I get to contribute to and the things I learn from them.\u00a0 Not every team gets the opportunity to work on so many different projects, in many different languages and on many different platforms. Recently, I enjoyed the luxury of helping to integrate the Firefox Lockwise Android and iOS applications<\/a> with Glean<\/a>.\u00a0 Getting to work alongside another team is always fun and this was especially true with the Lockwise Team due to their obvious energy and helpful nature.<\/p>\n

After integrating Glean with any new app, one of the most important steps we do is to validate that the integration is working and that we are getting the data we expect.\u00a0 This allows us to ensure that consuming applications can rely on the data provided by Glean and also allows us to find some unusual bugs. This blog post is going to focus on a couple of the interesting bugs and surprises that I encountered while validating the Glean data coming from the Lockwise mobile apps.<\/p>\n

The first interesting and\/or troubling thing that I discovered through my validation of the Lockwise on Android integration was that the Android Autofill service doesn\u2019t have the same Activity lifecycle that a normal application does because it is invoked by the OS as a service.\u00a0 Since Glean relies on certain Activity lifecycle events that weren\u2019t being generated when the app was invoked for the Autofill service, and since a lot of Lockwise\u2019s usage is performed in this way, we were seeing a lot less usage than compared to the legacy telemetry. Legacy telemetry was being sent in a more explicit way from the application, so it wasn\u2019t affected by this particular issue.\u00a0 This also affected the overall client counts, because Glean was only counting the clients that launched the app, and not the ones that were just using the Autofill service. This amounted to about 26 times as many legacy pings as baseline pings for the sample group. This same trend was visible no matter how the sample was segregated.<\/p>\n

Below is an example of how many more pings we were seeing from legacy telemetry (the blue line) than Glean (the green line) for Lockwise on Android.<\/p>\n

\"\"<\/p>\n

The next surprising thing I ran into was the lack of data in legacy telemetry.\u00a0 It\u2019s really hard to compare things when you don\u2019t have anything to compare to. This was a result of me trying to see what a side-by-side comparison of the length of times that a user had the app open, or more specifically, running in the foreground of the device.\u00a0 Glean data provided a nice distribution of usage which looked like it could provide some insights into how users were interacting with the app. Unfortunately, legacy telemetry let me down here when I discovered that each and every ping had a duration of zero.<\/p>\n

Below is an example of the data I could see through Glean, but unfortunately had nothing from legacy telemetry which to validate it against.<\/p>\n

\"\"<\/p>\n

All of this work was still worthwhile, despite the differences, as it exposed some things with Glean that we hadn\u2019t considered, such as using Glean from a service rather than an application and how the difference in lifecycles affected ping submission.\u00a0 Through my investigation, I also was able to see firsthand the improvements that Glean provided by just providing information that was missing from legacy telemetry.<\/p>\n

That was how the Android investigation went, and I followed it up by performing basically the same investigation for the Lockwise on iOS application.\u00a0 After seeing how many surprising things surfaced during the Android investigation, I went into this one with fewer expectations.<\/p>\n

Once again, I started with looking at the usage information: the ping counts and client counts between Glean and legacy telemetry.\u00a0 This turned out looking much more like I had hoped with just slightly less Glean pings than legacy pings, and client counts that tracked very closely for the sample group.\u00a0 Well, it looks like the Autofill service on iOS is handled differently since we seem to see about the same usage from Glean as we do from legacy telemetry.<\/p>\n

\"\"<\/p>\n

This goes to illustrate that the differences in platforms are an important consideration in how we build and test Glean.\u00a0 Similar functionality between Android and iOS doesn\u2019t always mean similar behavior for a cross-platform library like Glean.<\/p>\n

Not everything with the iOS investigation went as well.\u00a0 When I went to look at the length of time that the app was in the foreground, Glean had the expected data, but once again, legacy telemetry was lacking any information about duration.\u00a0 One nice thing about the Lockwise on iOS foreground duration information that I found was that it was comparable to the information I got from the Android investigation. Both showed similar usage from both Android and iOS.\u00a0 There were still some differences, but since we aren\u2019t getting all of the pings from the Android version it\u2019s a little hard to compare them properly.<\/p>\n

Here\u2019s what the foreground duration looks like from Lockwise-iOS:<\/p>\n

\"\"<\/p>\n

At times during the investigation I felt like I was comparing apples to oranges, but there were valuable things learned and positive validation of data coming in.\u00a0 It was a solid reminder that we are still learning about use-cases for Glean and that there are differences between platforms that must be considered by cross-platform SDK\u2019s like Glean.\u00a0 We walk away from this investigation with some action items on things to improve and some new information about how Glean works a little differently between platforms. All in a day\u2019s work…<\/p>\n","protected":false},"excerpt":{"rendered":"

(\u201cThis Week in Glean\u201d is a series of blog posts that the Glean Team at Mozilla is using to try to communicate better about our work. They could be release … Read more<\/a><\/p>\n","protected":false},"author":1757,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[315988,448297],"tags":[448297,446082],"coauthors":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/posts\/137"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/users\/1757"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/comments?post=137"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/posts\/137\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/media?parent=137"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/categories?post=137"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/tags?post=137"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/coauthors?post=137"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}