{"id":457,"date":"2015-09-21T20:58:50","date_gmt":"2015-09-21T20:58:50","guid":{"rendered":"http:\/\/blog.mozilla.org\/webqa\/?p=457"},"modified":"2015-09-21T20:58:50","modified_gmt":"2015-09-21T20:58:50","slug":"selenium-conference-2015-wrap-up-notes","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/fxtesteng\/2015\/09\/21\/selenium-conference-2015-wrap-up-notes\/","title":{"rendered":"Selenium Conference 2015 Wrap-up Notes"},"content":{"rendered":"

This year, I set out to attend Selenium Conference 2015<\/a> with some specific goals:<\/strong><\/p>\n

    \n
  1. See which JavaScript framework(s)\/librar(y|ies) are being used and recommended for “full-stack” testing (as a potential complement or replacement to our Python stack).\u00a0 Check out an earlier post on InternJS<\/a> for our goals and challenges.<\/li>\n
  2. See which mobile-testing strategies + tooling are being used.<\/li>\n
  3. Look into what folks are using Docker<\/a> for, and how we might apply it.<\/li>\n
  4. Cloud, cloud, cloud (and, more specifically, for us) + AWS<\/a> – what are companies and organizations of all sizes and needs using it for, testing-wise, and what are the considerations and gotchas, etc., as well as how testing considerations change in the DevOps model that often accompanies cloud.<\/li>\n
  5. Which open-source tools can we use + hopefully integrate, to get performance + accessibility testing, along with our functional tests?\u00a0 (Particularly, as close to Real User Monitoring<\/a> as we can get, without a third-party, expensive, hosted solution – or in addition.)<\/li>\n
  6. Catch up on what the core committers of the Selenium project<\/a> are doing, thinking, and where the project is next headed.<\/li>\n
  7. General information-seeking + evangelizing, particularly around open-source offerings, both in terms of what we could potentially use, but also sharing what we’ve done, are working on, and can share with the community.<\/li>\n<\/ol>\n

    Talks I attended and particularly enjoyed were:<\/strong><\/p>\n

      \n
    1. Simon Stewart’s keynote, entitled: Selenium: State of the Union – video<\/a> (no slides available)<\/li>\n
    2. Curing Imposter Syndrome – video<\/a>, slides<\/a><\/li>\n
    3. Mobile End-to-End Testing at Scale: Stable, Useful, Easy – Pick Three – video<\/a>, slides<\/a><\/li>\n
    4. Distributed Automation using Selenium Grid\/AWS\/Auto-scaling – video<\/a>, slides<\/a><\/li>\n
    5. Docker Selenium: Getting Started – video<\/a>, slides<\/a><\/li>\n<\/ol>\n

      A few key takeaways:<\/strong><\/p>\n

        \n
      1. Very common among many companies for monitoring and continuous deployment were: statsd<\/a>, graphite<\/a>, Jenkins CI<\/a>, and Travis CI<\/a>.<\/li>\n
      2. If not optimized, AWS can be _very_ expensive, and you can very easily end up wasting paid-for computing cycles\/resources.<\/li>\n
      3. Docker looks very useful if you only need to test (using Selenium Grid<\/a>) on Linux, primarily – it’s less clear with Windows + Mac.\u00a0 The particular talk on this used Ruby as its language, although it shouldn’t be too much work to do in Python.<\/li>\n
      4. All the core committers I spoke with, and other industry peers as well, recommended “just using” WebDriverJS<\/a> if we’re going to continue experimenting with JavaScript-based, developer-integration level testing.\u00a0 Yahoo! has also used Protractor<\/a> (for Angular apps) + CasperJS<\/a>, along with WebDriverJS as its core, with good results.<\/li>\n
      5. Lots of discussion around mobile testing, and while most of the folks testing mobile do so using Appium<\/a> (as do we), the needs vary widely, as with good monitoring and responsive design, there isn’t always a need to do full end-to-end tests across all devices.\u00a0 This also depends on whether you’re testing a mobile website or a native app on Android or iOS, of course.\u00a0 Appium 1.5 is coming soon, we learned, and will be a nice, clean release with a lot of fixes + features.<\/li>\n
      6. Yahoo! has used sitespeed.io<\/a> to drive webpagetest.org<\/a>; it looks really useful, and thankfully seems kept up-to-date.<\/li>\n<\/ol>\n

        Action Items:<\/strong><\/p>\n

          \n
        1. Catch-up with the remainder of the Selenium Conference 2015 talks<\/a> I wasn’t able to attend.<\/li>\n
        2. Look further into the potential use-case(s) for Docker, test-infrastructure wise.<\/li>\n
        3. Investigate integrating a Selenium proxy (such as browsermob-proxy<\/a>) with HAR files and potentially Apache\/Nginx server logs, to augment our HTML Reports with richer, more timely information to help better-report and debug problems (heavily re-inspired by the Facebook talk!)<\/li>\n
        4. Look into the possibilities of scaling our tests + increasing performance using AWS\/cloud infra, while balancing cost considerations.<\/li>\n
        5. Look into evaluating and potentially integrating visual-comparison testing tools, such as Applitools Eyes<\/a>.<\/li>\n
        6. Evangelize – through blog posts, meetups, etc., more of what Web QA and Mozilla are up to – particularly for our peers and community’s benefit.<\/li>\n<\/ol>\n

          For more wrap-up posts on last week’s separate, but follow-on Web QA team meetings, look here, as well: https:\/\/blog.mozilla.org\/webqa\/category\/meetings\/<\/a><\/p>\n

          Keep an eye on this blog for future posts; we’ll be posting a little more often than we have in recent months!<\/p>\n

          – Stephen<\/a><\/p>\n

           <\/p>\n","protected":false},"excerpt":{"rendered":"

          This year, I set out to attend Selenium Conference 2015 with some specific goals: See which JavaScript framework(s)\/librar(y|ies) are being used and recommended for “full-stack” testing (as a potential complement … Read more<\/a><\/p>\n","protected":false},"author":512,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[53,39],"tags":[25193],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/posts\/457"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/users\/512"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/comments?post=457"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/posts\/457\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/media?parent=457"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/categories?post=457"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/fxtesteng\/wp-json\/wp\/v2\/tags?post=457"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}