{"id":310,"date":"2013-07-16T10:00:49","date_gmt":"2013-07-16T10:00:49","guid":{"rendered":"http:\/\/blog.mozilla.org\/services\/?p=310"},"modified":"2013-07-16T17:07:40","modified_gmt":"2013-07-16T17:07:40","slug":"heka-0-3-released","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/services\/2013\/07\/16\/heka-0-3-released\/","title":{"rendered":"Heka 0.3 released"},"content":{"rendered":"<p>Those of us here on Mozilla Service&#8217;s Heka team were very pleased by the positive response and interest generated by our initial <a title=\"Introducing Heka\" href=\"http:\/\/blog.mozilla.org\/services\/2013\/04\/30\/introducing-heka\/\">announcement<\/a> about the project. And we&#8217;re even more pleased by the fact that some of you out there have decided to help out, contributing doc tweaks, bug fixes, and, in some cases, completely new plugins back to the Heka core. All the activity has kept us inspired, and we&#8217;ve landed a huge number of fixes and improvements ourselves since then. We&#8217;re happy to be rolling these out in a new Heka 0.3 release.<\/p>\n<p>A full list of what&#8217;s new in this release can be found in the <a href=\"https:\/\/github.com\/mozilla-services\/heka\/blob\/v0.3.0\/CHANGES.txt\">changelog<\/a>, but here are some of the bigger features:<\/p>\n<ul>\n<li><strong>ElasticSearch output:<\/strong> We had just decided that we wanted to write Heka message data out to <a title=\"ElasticSearch\" href=\"http:\/\/www.elasticsearch.org\/\">ElasticSearch<\/a> (so we could search through our data using a <a title=\"Kibana\" href=\"http:\/\/three.kibana.org\/\">Kibana<\/a> dashboard) when we received a pull request from <a title=\"@tlrx\" href=\"https:\/\/github.com\/tlrx\/\">Tanguy Leroux<\/a> providing exactly <a title=\"ElasticSearchOutput\" href=\"http:\/\/http:\/\/hekad.readthedocs.org\/en\/latest\/configuration.html#elasticsearchoutput\">that<\/a>. The screenshot below is of a Kibana dashboard. It is displaying a histogram of the 10 (anonymized) Firefox Sync users who received the most 503 HTTP response codes over a specific period of time, extracted by Heka from our load balancer log files.<a href=\"http:\/\/blog.mozilla.org\/services\/files\/2013\/07\/heka-kibana.png\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-311 aligncenter\" alt=\"heka-kibana\" src=\"http:\/\/blog.mozilla.org\/services\/files\/2013\/07\/heka-kibana.png\" width=\"458\" height=\"288\" \/><\/a><\/li>\n<li><strong>Restartable plugins:<\/strong> It is now possible to specify any Heka input, filter, or output plugin as <a href=\"http:\/\/hekad.readthedocs.org\/en\/latest\/configuration.html#configuring-restarting-behavior\">restartable<\/a>, so it will reinitialize itself and start over when encountering an error. This is especially useful for plugins that require persistent connections to external services, as it allows them to reconnect. You can also set them to back off exponentially up to a user-defined cap, or add some timing jitter to prevent several reconnection attempts from happening simultaneously.<\/li>\n<li><strong>Resume-from-location log file parsing:<\/strong> When shutting down, <a href=\"http:\/\/hekad.readthedocs.org\/en\/latest\/configuration.html#logfileinput\">LogfileInput<\/a> will note where it stopped parsing a log file, and will try to pick up from the same location when it restarts.<\/li>\n<li><strong>Nagios output:<\/strong> If you use <a href=\"http:\/\/www.nagios.org\/\">Nagios<\/a> for monitoring, you can now use the <a href=\"http:\/\/hekad.readthedocs.org\/en\/latest\/configuration.html#nagiosoutput\">NagiosOutput<\/a> plugin to generate notifications triggered by Heka messages. Combine this with the ability to do arbitrary data processing in Heka&#8217;s dynamic Lua filters, and it becomes very easy to set up ad-hoc notifications for specific targeted events.<\/li>\n<li><strong>Improved text parsing:<\/strong> We&#8217;ve moved the regular expression match group capturing functionality out of the router and into a <a href=\"http:\/\/hekad.readthedocs.org\/en\/latest\/configuration.html#loglinedecoder\">decoder<\/a>, so it won&#8217;t slow down routing of messages that don&#8217;t use capture groups. We also managed to add some timezone-shifting functionality, for cases where a non-UTC time zone is used but not specified in the timestamps.<\/li>\n<li><strong>HTTP input:<\/strong> Thanks to an initial effort by <a title=\"@linkdd\" href=\"https:\/\/github.com\/linkdd\/\">David Delassus<\/a>, we&#8217;ve now got an <a href=\"http:\/\/hekad.readthedocs.org\/en\/latest\/configuration.html#httpinput\">HttpInput<\/a> plugin that will make HTTP requests and turn the resulting response bodies into Heka messages. You&#8217;ll need a custom Lua filter to parse the results and extract useful data, at least until the <a href=\"https:\/\/github.com\/mozilla-services\/heka\/issues\/309\">helpful<\/a> <a href=\"https:\/\/github.com\/mozilla-services\/heka\/issues\/310\">decoders <\/a>that we have under development are ready to take over that job for you.<\/li>\n<li><strong>Cloudwatch input &amp; output:<\/strong> We&#8217;ve added plugins to get data <a title=\"CloudwatchInput\" href=\"https:\/\/github.com\/mozilla-services\/heka-mozsvc-plugins\/blob\/dev\/docs\/config.rst#cloudwatch-input\">out of<\/a> and <a title=\"CloudwatchOutput\" href=\"https:\/\/github.com\/mozilla-services\/heka-mozsvc-plugins\/blob\/dev\/docs\/config.rst#cloudwatch-output\">into<\/a> Amazon&#8217;s Cloudwatch metrics service. They&#8217;re not in the Heka core, but they&#8217;re in the Mozilla Services <a title=\"heka-mozsvc-plugins\" href=\"https:\/\/github.com\/mozilla-services\/heka-mozsvc-plugins\">repository<\/a> of custom Heka plugins and are available in the released binaries.<\/li>\n<li><strong>New mailing list:<\/strong> There&#8217;s a new, dedicated <a title=\"Heka mailing list\" href=\"https:\/\/mail.mozilla.org\/listinfo\/heka\">Heka mailing list<\/a> for announcements about changes to configuration options, Heka behavior, and anything else that might impact running Heka servers. Anyone interested in Heka should check it out!<\/li>\n<\/ul>\n<p>As you can see, that&#8217;s a lot of progress. Big thanks to the Heka team and everyone who sent in patches, bug reports, and suggestions &#8211; keep them coming!<\/p>\n<p>Heka is improving rapidly, but it&#8217;s still best suited for early adopters at this point. If you&#8217;re interested in rolling your sleeves up and digging in, please feel free to check out the <a title=\"Heka v0.3 binaries\" href=\"https:\/\/github.com\/mozilla-services\/heka\/releases\/tag\/v0.3.0\">binaries<\/a>, the <a href=\"https:\/\/github.com\/mozilla-services\/heka\">source code<\/a>, and the <a title=\"hekad documentation\" href=\"http:\/\/hekad.readthedocs.org\/en\/latest\/\">documentation<\/a>. And don&#8217;t forget to join the <a title=\"Heka mailing list\" href=\"https:\/\/mail.mozilla.org\/listinfo\/heka\">mailing list<\/a>, and to drop in to the #heka channel on irc.mozilla.org to ask questions or share your experiences.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Those of us here on Mozilla Service&#8217;s Heka team were very pleased by the positive response and interest generated by our initial announcement about the project. And we&#8217;re even more pleased by the fact that some of you out there &hellip; <a class=\"go\" href=\"https:\/\/blog.mozilla.org\/services\/2013\/07\/16\/heka-0-3-released\/\">Continue reading<\/a><\/p>\n","protected":false},"author":598,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30944],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/services\/wp-json\/wp\/v2\/posts\/310"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/services\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/services\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/services\/wp-json\/wp\/v2\/users\/598"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/services\/wp-json\/wp\/v2\/comments?post=310"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/services\/wp-json\/wp\/v2\/posts\/310\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/services\/wp-json\/wp\/v2\/media?parent=310"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/services\/wp-json\/wp\/v2\/categories?post=310"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/services\/wp-json\/wp\/v2\/tags?post=310"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}