{"id":200,"date":"2020-06-03T12:05:44","date_gmt":"2020-06-03T12:05:44","guid":{"rendered":"https:\/\/blog.mozilla.org\/data\/?p=200"},"modified":"2020-06-03T12:29:02","modified_gmt":"2020-06-03T12:29:02","slug":"this-week-in-glean-the-glean-sdk-and-ios-application-extensions-or-a-tale-of-two-sandboxes","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/data\/2020\/06\/03\/this-week-in-glean-the-glean-sdk-and-ios-application-extensions-or-a-tale-of-two-sandboxes\/","title":{"rendered":"This Week in Glean: The Glean SDK and iOS Application Extensions, or A Tale of Two Sandboxes"},"content":{"rendered":"

(\u201cThis Week in Glean\u201d is a series of blog posts that the Glean Team at Mozilla is using to try to communicate better about our work. They could be release notes, documentation, hopes, dreams, or whatever: so long as it is inspired by Glean. You can find an index of all TWiG posts online<\/a>.)<\/p>\n

Recently, I had the pleasure of working with our wonderful iOS developers here at Mozilla in instrumenting Lockwise<\/a>, one of our iOS applications, with the Glean SDK.\u00a0 At this point, I\u2019ve already helped integrate it with several other applications, all of which went pretty smoothly, and Lockwise for iOS held true to that.\u00a0 It wasn\u2019t until later, when unexpected things started happening, that I realized something was amiss…<\/p>\n

Integrating the Glean SDK with a new product is a fairly straightforward process<\/a>.\u00a0 On iOS, it amounts to adding the dependency via Carthage<\/a>, and adding a couple of build-steps to get it to do its thing.\u00a0 After this is done, we generally smoke test the data using the built in debugging tools<\/a>.\u00a0 If everything looks good, we submit a request for data review<\/a> for collecting the new metrics.\u00a0 Once a data steward has signed off on our request to collect new data, we can then release a new version of the application with its Glean SDK powered telemetry.\u00a0 Finally, we collect a few weeks of data to validate that everything looks good, such as user counts, distribution of locales, and we look for anything that might indicate that the data isn\u2019t getting collected like we expected, such as holes in sequence numbers or missing fields.\u00a0 In Lockwise for iOS\u2019s case, all of this went just as expected.<\/p>\n

One part of the Glean SDK integration that I haven\u2019t mentioned yet is enabling the application in our data ingestion pipeline via the probe-scraper<\/a> so that we can accept data from it.\u00a0 On iOS, the Glean SDK makes use of the application bundle identifier<\/a> to uniquely identify the app to our pipeline, so enabling the app means letting the pipeline know about this id so that it won\u2019t turn away the data.\u00a0 This identifier also determines the table that the data ultimately ends up in, so it\u2019s a key identifier in the process.<\/p>\n

So, here\u2019s where I learned something new about iOS architecture, especially as it relates to embedded application extensions<\/a>.\u00a0 Application extensions are a cool and handy way of adding additional features and functionality to your application in the Apple ecosystem.\u00a0 In the case of Lockwise, they are using a form of extension that provides credentials to other applications.\u00a0 This allows the credentials stored in Lockwise to be used to authenticate in websites and other apps installed on the device.\u00a0 I knew about extensions but hadn\u2019t really worked with them much until now, so it was pretty interesting to see how it all worked in Lockwise.<\/p>\n

Here\u2019s where a brick smacks into the story.\u00a0 Remember that bundle identifier that I said was used to uniquely identify the app?\u00a0 Well, it turns out that application extensions in iOS modify this a bit by adding to it to uniquely identify themselves!\u00a0 We realized this when we started to see our pipeline reject this new identifier, because it wasn\u2019t an exact match for the identifier that we expected and had allowed through.\u00a0 The id we expected was org-mozilla-ios-lockbox<\/a>, but the extension was reporting org-mozilla-ios-Lockbox-CredentialProvider.\u00a0 Using a different bundle identifier totally makes sense, since they run as a separate process within their own application sandbox container.\u00a0 The OS needs to see them differently because an extension can run even if the base application isn\u2019t running.\u00a0 Unfortunately, the Glean SDK is purposefully built to not care about, or even know about different processes so we had a bit of a blind spot in the application extension.\u00a0 Not only that, but remember I mentioned that the extension\u2019s storage container is a separate sandbox from the base application?\u00a0 Well, since the extension runs in a different process from the base application, and it has a separate storage, the Glean SDK running in the extension acted just like the extension was a completely separate application.\u00a0 With separate storage, it happily generates a different unique identifier for the client, which does not match the id generated for the base application.\u00a0 So there was no way to attribute the information in the extension to the base application that contained it because the ingestion pipeline saw these as separate applications with no way to associate the client ids between the two.\u00a0 These were two sandboxes that just couldn\u2019t interact with each other.\u00a0 To be fair, Apple does provide a way to share data between extensions and applications, but it requires creating a completely separate shared sandbox, and this doesn\u2019t solve the problem that the same Glean SDK instance just shouldn\u2019t be used directly by multiple processes at the same time.<\/p>\n

Well, that wasn\u2019t ideal, to say the least, so we began an investigation<\/a> to determine what course of action we should (or could) take.\u00a0 We went back and forth over the details but ultimately we determined that the Glean SDK shouldn\u2019t know about processes and that there wasn\u2019t much we could do aside from blocking it from running in the extensions and documenting the fact that it was up to the Glean SDK-using application to ensure that metrics were only collected by the main process application.\u00a0 I was a bit sad that there wasn\u2019t much we could do to make the user-experience better for Glean SDK consumers, but sometimes you just can\u2019t predict the challenges you will face when implementing a truly cross-platform thing.\u00a0 I still hold out hope that a way will open up to make this easier, but the lesson I learned from all of this is that sometimes you can\u2019t win but it\u2019s important to stick to the design and do the best you can.<\/p>\n","protected":false},"excerpt":{"rendered":"

(\u201cThis Week in Glean\u201d is a series of blog posts that the Glean Team at Mozilla is using to try to communicate better about our work. They could be release … Read more<\/a><\/p>\n","protected":false},"author":1757,"featured_media":197,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[448297],"tags":[448297,27815,446082],"coauthors":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/posts\/200"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/users\/1757"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/comments?post=200"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/posts\/200\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/media\/197"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/media?parent=200"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/categories?post=200"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/tags?post=200"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mozilla.org\/data\/wp-json\/wp\/v2\/coauthors?post=200"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}