{"id":395,"date":"2022-02-25T13:45:55","date_gmt":"2022-02-25T13:45:55","guid":{"rendered":"https:\/\/blog.mozilla.org\/data\/?p=395"},"modified":"2022-02-25T13:45:55","modified_gmt":"2022-02-25T13:45:55","slug":"this-week-in-glean-your-personal-glean-data-pipeline","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/data\/2022\/02\/25\/this-week-in-glean-your-personal-glean-data-pipeline\/","title":{"rendered":"This Week in Glean: Your personal Glean data pipeline"},"content":{"rendered":"

(\u201cThis Week in Glean\u201d is a series of blog posts that the Glean Team at Mozilla is using to try to communicate better about our work. They could be release notes, documentation, hopes, dreams, or whatever: so long as it is inspired by Glean.) All \u201cThis Week in Glean\u201d blog posts are listed in the TWiG index<\/a> (and on the Mozilla Data blog<\/a>).<\/p>\n

\n

On February 11th, 2022 we hosted a Data Club Lightning Talk session. There I presented my small side project of setting up a minimal data pipeline & data storage for Glean.<\/p>\n

The premise:<\/p>\n

\n

Can I build and run a small pipeline & data server to collect telemetry data from my own usage of tools?<\/p>\n<\/blockquote>\n

To which I can now answer: Yes, it\u2019s possible. The complete ingestion server is a couple hundred lines of Rust code. It\u2019s able to receive pings conforming to the Glean ping schema, transform them and store it into an SQLite database. It\u2019s very robust, not crashing once on me (except when I created an infinite loop within it).<\/p>\n

You can watch the lightning talk here:<\/p>\n

<\/p>\n