{"id":495,"date":"2016-07-07T01:18:22","date_gmt":"2016-07-07T01:18:22","guid":{"rendered":"http:\/\/blog.mozilla.org\/webqa\/?p=495"},"modified":"2016-07-07T01:18:22","modified_gmt":"2016-07-07T01:18:22","slug":"tough-lessons-learned-from-integrating-docker-zap-cli-and-jenkins","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/fxtesteng\/2016\/07\/07\/tough-lessons-learned-from-integrating-docker-zap-cli-and-jenkins\/","title":{"rendered":"(Tough) Lessons learned from integrating Docker, ZAP-CLI, and Jenkins"},"content":{"rendered":"<p>I learned a great deal &#8212; both about technology and approaches in using it &#8212; while I worked through <a href=\"https:\/\/wiki.mozilla.org\/QA\/Execution\/Web_Testing\/Goals\/2016\/Q2#Stephen\">last quarter&#8217;s goal<\/a> of getting a Dockerized OWASP-ZAP scanning instance stood up in Jenkins, and running against a live server.<\/p>\n<p>Both for the sake of the original two blog posts&#8217; lengths, as well as meeting my Q2 goal (and give myself a much-needed breather), I decided to collect my thoughts along the way, and blog about them.<\/p>\n<p>So now I&#8217;d like to share those lessons learned from my first milestone of <a href=\"https:\/\/blog.mozilla.org\/webqa\/2016\/05\/11\/docker-owasp-zap-part-one\/\">getting the ZAP-CLI running in Docker<\/a>, to the end goal of having the <a href=\"https:\/\/blog.mozilla.org\/webqa\/2016\/06\/28\/dockerized-owasp-zap-security-scanning-in-jenkins-part-two\/\">Dockerized ZAP-CLI running inside Jenkins<\/a>, scanning against a live website.<\/p>\n<h3>The Good:<\/h3>\n<p>Open, public <a href=\"https:\/\/github.com\/stephendonner\/docker-zap\">docker-zap GitHub repo<\/a> which allowed me to:<\/p>\n<ul>\n<li>file and address <a href=\"https:\/\/github.com\/stephendonner\/docker-zap\/issues?utf8=%E2%9C%93&amp;q=is%3Aissue\">Issues<\/a> along the way, both for my own progress, as well as for visibility (thanks, <a href=\"https:\/\/github.com\/Grunny\">Grunny!<\/a>  I&#8217;ll be circling back soon, promise!)<\/li>\n<li>use source control not only for myself, but to hopefully attract and encourage collaboration (patches)<\/li>\n<li>quickly integrate with Jenkins, when the time came<\/li>\n<\/ul>\n<p>Blogged the step-by-step process of getting up and running: <a href=\"https:\/\/blog.mozilla.org\/webqa\/2016\/05\/11\/docker-owasp-zap-part-one\/\">part one<\/a> and <a href=\"https:\/\/blog.mozilla.org\/webqa\/2016\/06\/28\/dockerized-owasp-zap-security-scanning-in-jenkins-part-two\/\">part two<\/a>.  This turned out to be more useful for myself than even I imagined:<\/p>\n<ul>\n<li>recreating the instructions step-by-step, many times, helped me both get a deeper understanding, as well as provided me with a handy reference to check against, towards the end, as I made simple mistakes<\/li>\n<\/ul>\n<h3>NOW, THE BAD:<\/h3>\n<ul>\n<li>Thinking it to be overwhelming, I didn\u2019t start trying Docker with ZAP-CLI in Jenkins, on the target (RedHat Enterprise Linux) system.  Instead, I learned enough Docker to get the ZAP-CLI going on my OS X laptop, but had to (re)learn quite a bit of Linux (OS-level architectural differences + commands [<a href=\"https:\/\/www.linux.com\/blog\/how-use-sudo-and-su-commands-linux-introduction\"><code>sudo<\/code> vs. <code>su<\/code><\/a>, among them] + Docker-behavior changes), once I got to the integration phase<\/li>\n<li>Huge platform differences:<\/li>\n<ol>\n<li>Getting the <a href=\"https:\/\/docs.docker.com\/v1.10\/engine\/reference\/commandline\/daemon\/\">Docker Daemon<\/a> and <a href=\"https:\/\/docs.docker.com\/engine\/reference\/api\/docker_remote_api\/\">Remote\/REST API<\/a> (not the containers themselves, as much) up and running &#8212; securely &#8212; is very different on Linux and Mac<\/li>\n<li><a href=\"https:\/\/access.redhat.com\/documentation\/en\/red-hat-enterprise-linux-atomic-host\/7\/container-security-guide\/chapter-6-docker-selinux-security-policy\">SELinux<\/a> was enabled by default on Linux (Fedora)<\/li>\n<\/ol>\n<li>I spent a lot of time reading about (and playing with a few of) the <a href=\"https:\/\/jenkins.io\/solutions\/docker\/\">various Docker plugins for Jenkins<\/a>, none of which I ended up using<\/li>\n<ol>\n<li>The simplest plugin, <a href=\"https:\/\/wiki.jenkins-ci.org\/display\/JENKINS\/Docker+build+step+plugin\">Docker Build Step Plugin<\/a>, doesn&#8217;t support (because the <a href=\"https:\/\/docs.docker.com\/engine\/reference\/api\/docker_remote_api\/\">Docker Remote API<\/a> itself doesn&#8217;t) the <a href=\"https:\/\/docs.docker.com\/engine\/reference\/run\/\">docker run<\/a> command\n<li>I didn&#8217;t get my integrated work in a publicly-hosted instance soon enough.  This made it <em><strong>much<\/strong><\/em> more difficult for others to contextualize issues I was facing, and help make or suggest changes, live.  Instead, we had to:\n<ol>\n<li>Share code snippets<\/li>\n<li>Ask and try things like &#8220;What does <code>cat \/var\/run\/docker.socket<\/code> yield?&#8221;<\/li>\n<li>Share screenshots<\/li>\n<\/ol>\n<li>Red Herrings<\/li>\n<ol>\n<li>Hard to understand which user (jenkins vs. docker, primarily) was running which command<\/li>\n<li>Due to the wording on the \u201cConfiguration\u201d portion of the <a href=\"https:\/\/wiki.jenkins-ci.org\/display\/JENKINS\/Docker+build+step+plugin\">plugin\u2019s page<\/a> which mentioned slaves, I didn\u2019t bother to specify the Docker REST API URL in the Jenkins global config (didn\u2019t think it needed it)<\/li>\n<ol>\n<li>Filed an <em>INVALID<\/em> Issue about that: <a href=\"https:\/\/issues.jenkins-ci.org\/browse\/JENKINS-35322\">https:\/\/issues.jenkins-ci.org\/browse\/JENKINS-35322<a \/><\/li>\n<\/ol>\n<\/ol>\n<\/ol>\n<\/ul>\n<h3>FINALLY, THE UGLY:<\/h3>\n<p>The most basic thing I should have done upfront was to ensure that Docker would work on the intended RHEL-6 based Jenkins box, but <a href=\"https:\/\/access.redhat.com\/solutions\/1378023\">Docker isn&#8217;t supported on RHEL6&#8230;<\/a><\/p>\n<ul>\n<li>This shouldn&#8217;t have happened; I should have known to explicitly search for two basic terms, &#8220;RHEL6 Docker&#8221; and saved myself much lost time and shame<\/li>\n<\/ul>\n<h3>Good lessons learned:<\/h3>\n<ul>\n<li>Get as close to your target system as early as possible<\/li>\n<li>Get a public repo out there, if you can<\/li>\n<li>Just as important (and potentially even moreso), get an accessibe-to-others instance up and running as soon as possible.  It\u2019ll also help inform you as to whether approaches should change, which can save a lot of time and headache down the road, instead of trying to retrofit and reconcile two or more disparate technologies towards the end<\/li>\n<\/ul>\n<p>Especially when &#8220;just trying to get it working,&#8221; a huge lesson well-learned is: I spent too much time just trying to \u201cget the plugin to work,\u201d rather than understanding and even trying to get things up and running without it (which ended up both being cleaner, code-wise, as well as helped me learn what I was actually setting up).  Specifically, if &#8212; without any plugins, you can &#8211;:<\/p>\n<ul>\n<li>expose the API and its functionality you need<\/li>\n<li>on the host you need<\/li>\n<li>so others can access it and wrap\/work on their own workflows, using the above<\/li>\n<\/ul>\n<p>So I&#8217;m not misunderstood: plugins are fine, but make sure you understand which problem(s) they solve that you &#8212; and potentially your peers &#8212; can&#8217;t solve.  And, furthermore, that ages-old <a href=\"https:\/\/en.wikipedia.org\/wiki\/KISS_principle\">KISS principle<\/a> is (still) there for a good reason!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I learned a great deal &#8212; both about technology and approaches in using it &#8212; while I worked through last quarter&#8217;s goal of getting a Dockerized OWASP-ZAP scanning instance stood &hellip; <a class=\"go\" href=\"https:\/\/blog.mozilla.org\/fxtesteng\/2016\/07\/07\/tough-lessons-learned-from-integrating-docker-zap-cli-and-jenkins\/\">Read more<\/a><\/p>\n","protected":false},"author":512,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[228,69],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/posts\/495"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/users\/512"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/comments?post=495"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/posts\/495\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/media?parent=495"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/categories?post=495"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/tags?post=495"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}