A New Focus on Security in the Web Console

Garrett Robinson

Web developers need better tools to help them debug security issues. The Web Console, part of the Firefox Developer Tools, shows errors and warnings filtered into different categories. Firefox 23 adds a new category of messages to the Web Console: Security messages.

Toggle buttons for categories of messages in the Web Console

Toggle buttons for categories of messages in the Web Console

The Security toggle button and messages are red to warn developers, since some of these messages indicate that your site has a security vulnerability.

Once we had a dedicated place for security messages, we had to decide what kinds of issues should be reported to developers. Ivan Alagenchev, a security engineering intern, spent the summer improving security reporting to fulfill the following goals:

  1. Warn developers about altered site behavior that is due to a security feature (for example, resource loads blocked by the Mixed Content Blocker or the Same Origin Policy).
  2. Warn developers about mistakes made in implementing security features (for example, using deprecated CSP headers, or mistyping an HSTS header).
  3. Warn developers about common security risks (for example, putting password fields on insecure pages).

Here are example screenshots of some of the new Security messages:

Errors for blocked mixed content in the Web Console.

Warnings for loading mixed content

Warning for detected password field on an insecure page.

Warning for detected password field on an insecure page.

These specific messages are available to current Nightly users and will be part of upcoming stable releases.

While security should be of paramount importance to any developer, it is a complex subject that is not always part of a web developer’s education and often appears at inconvenient times. This new messaging helps developers find security-related problems early on in the development life cycle so they can be resolved quickly and effectively.

Additionally, these messages help educate developers about common issues in web security. Many of the new messages end with a “Learn More” link that takes you to a wiki with background information and advice for mitigating the security issue.

Bug 863874 is the meta-bug for logging relevant security messages to the Web Console. If you have more ideas for useful features like the ones discussed here, or are interested in contributing, check out the metabug and its dependencies!

Writing Minion Plugins

yboily

The following blog post is contributed by Yeuk Hon, an intern who has been with the Security Automation team at Mozilla over the summer. Today is his last day with Mozilla, and this post serves as a tutorial on how to write Minion Plugins. As an aside, I would also like to thank Yeuk Hon for his awesome work over the summer! – Yvan Boily

Hello, the Web! I am Yeuk Hon, a summer intern working with the Security Assurance team. In this blog post, I will go over how a Minion plugin works and how to write a Minion plugin that works for your tool. Before you dive into my blog post, I encourage you to read Yvan’s Introducing Minion if you are not familiar with Minion already.

To recap briefly, Minion was created to make a web platform where developers can kick off active vulnerability scans against their own sites. The ability to execute a Python script to invoke other tools and applications makes Minion powerful, easy to use and extend.

Minion Recap

Minion’s workflow is quite simple. A developer would come on Minion, select a plan for a site, click the scan button and then wait for the scan report to come back. A Minion plan is a JSON document containing a list of workflows. A workflow is a JSON hashtable (or dictionary in Python) that specifies which Minion plugin and what configuration parameters to use. In essence, you can run a scan using a single Minion plugin or multiple Minion plugins with multiple configurations.

Here is an example of a Minion plan:

[
   {
      "configuration": {},
      "description": "Check to see if Set-Cookie has HttpOnly and secure flag enabled",
      "plugin_name": "minion.plugins.setcookie.SetCookiePlugin"
   },
   {
      "configuration": {
         "auth": {
            "type": "basic",
            "username": "foo",
            "password": "bar"
         },
         "scan": true
      },
      "description": "Run an active scan using X scanner",
      "plugin_name": "minion.plugins.example.ExampleScanner"
   }
]

This example plan will use two plugins. The first plugin does not expect any additional configurations while the second plugin, “ExampleScanner” is told to do a scan and the scanner is given the basic auth login. Configuration parameters can vary from one plugin to another, but we will try to document common plugin configuration patterns. For example, minion-zap-plugin and minion-skipfish-plugin both use the same authentication configuration pattern as shown above.

Plugin execution

There are two types of plugins in general: blocking plugins and external process plugins. Blocking plugins are Python scripts that do not invoke external processes like nmap. External process plugins use external processes like nmap to do the actual scan work, but the Python script helps spawning, collecting and returning results back to the Minion backend.

Minion’s backend uses the Twisted framework to drive events. To keep track of states, Minion uses RabbitMQ as broker and celery workers for queuing, state bookkeeping. and tasks execution. Below is a simplified diagram showing the plugin configuration, activation, and completion in the backend.

minion-plugin

The main takeaways from the diagram are:

  1. Minion’s backend spawns a process using Python’s subprocess called minion-plugin-runner.
  2. The runner will invoke the plugin’s “do_start” method to run the plugin. A Blocking plugin is handled by calling Twisted’s “deferThread” and an external process plugins invokes an external process via Twisted’s “spawnProcess”.

If you have used Twisted before, you probably realized that method names like “do_start” are Twisted conventions. As a plugin author, knowing Twisted can be helpful and so we recommend you to check out this Twisted guide.

Example 1: SetCookiePlugin

That’s a lot of information to digest so let’s look at an actual plugin. We will continue with the Set-Cookie plugin we listed in our example Minion plan. This snippet shows part of the plugin, but you can find the full source code here.

import requests
from minion.plugins.base import BlockingPlugin

class SetCookiePlugin(BlockingPlugin):
    PLUGIN_NAME = "SetCookie"
    PLUGIN_VERSION = "0.1"

    FURTHER_INFO = [ {"URL": "http://msdn.microsoft.com/en-us/library/windows/desktop/aa384321%28v=vs.85%29.aspx", "Title": "MSDN - HTTP Cookies"} ]

    def do_run(self):
        r = requests.get(self.configuration['target'])
        if 'set-cookie' not in r.headers:
            return self.report_issues([
                {'Summary': "Site has no Set-Cookie header",
                'Description': "The Set-Cookie header is sent by the server in response to an HTTP request, which is used to create a cookie on the user's system.",
                'Severity': "Info",
                "URLs": [ {"URL": None, "Extra": None} ],
                "FurtherInfo": self.FURTHER_INFO}])
        else:
            # take care of cases where
            # (1) HttpOnly flag is not set,
            # (2) secure flag is not set
            # (3) both flags ARE set

We first set the name of the plugin and the version of the plugin. Since the list of references (we called them further info in plugin) are static, we can make them class static member variable at the class level. We will talk about what BlockingPlugin class does later, but for the meantime, all we have to know do for this plugin is to override the do_run method. We just check to see if “set-cookie” is in the response header and then report our observation back to Minion in Minion’s report format. If set-cookie is not present, there is no risk so the level of severity is hardcoded to Info.

The report scheme looks like this:

[
    {
        "Summary": "One sentence description of the issue (required)",
        "Description": "In-depth description of the issue and why the issue matters (required)",
        "Solution": "Mitigations (optional)",
        "Severity": "High/Medium/Low/Info/Error/Fatal (required)",
        "URLs": [
        {
            "URL": "http://target_site.com/path1",
            "Extra": "Extra information on why this particular URL is affected"
        }],
        "FurtherInfo": [
        {
            "URL": "http://reference1.com/",
            "Title": "Reference read title 1"
        }]
}
]

The URLs are also optional and used to indicate what parts of the site are affected by the same issue. The values of severity level are High, Medium, Low, Info, Error and Fatal. Solution is optional if the issue doesn’t need to offer a solution. For example, when severity level is Info you don’t need any solution.

To implement a plugin, there are a few things to do.

  • A plugin must be a class and the top of the class chain should be “minion.plugins.base.AbstractPlugin“
  • “do_configure“, “do_start“, “do_stop“ methods are implemented
  • use self.report_issue to return a list of issues in JSON format
  • the “self.configuration“ contains all the configuration parameters passed from Minion plan in addition to the site url (which is defined as self.configuration[‘target’]).

Plugin classes

We spoke earlier we generalized plugins into blocking and external process, Minion is shipped with “BlockingPlugin“ and “ExternaProcessPlugin“ out of the box for plugin authors to use:

  1. “minion.plugins.base.BlockingPlugin“ is used as a parent class for plugins written in pure Python that don’t require running a subprocess.
  2. * “minion.plugins.base.ExternalProcessPlugin“ is used as a parent class for plugins that require launching an external process.

You can create your own plugin type by subclassing “AbstractPlugin“ (or further subclass from “BlockingPlugin“ and “ExteranlProessPlugin“) if you have to.

Example 2: SetCookieScannerPlugin

Here is the full source code running a Go program which mirrors the SetCookie plugin we spoke earlier.

from minion.plugins.base import ExternalProcessPlugin

# Set-Cookie checker by running setcookie scanner written in Go
class SetCookieScannerPlugin(ExternalProcessPlugin):
    PLUGIN_NAME = "SetCookieScanner"
    PLUGIN_VERSION = "0.1"

    def do_start(self):
        scanner_path = self.locate_program("setcookie_scanner")
        if not scanner_path:
            raise Exception("Cannot find setcookie_scanner program.")

        self.stdout = ""
        self.stderr = ""

        # spawn by calling the executable and a list of args
        self.spawn(scanner_path, [self.configuration['target']])

    def do_procss_stdout(self, data):
        self.stdout += data

    def do_process_stderr(self, data):
        self.stderr += data

    def do_process_ended(self, process_status):
        if self.stopping and process_statsu == 9:
            self.report_finish("STOPPED")
        elif process_status == 0:
            # try to convert the JSON outputs in stdout
            stdouts = self.stdout.split('\n')
            minion_issues = []
            for stdout in stdouts:
                try:
                    minion_issues.append(json.loads(stdout))
                except ValueError:
                    logging.info(stdout)
            pass

        self.report_issues(minion_issues)
        self.report_finish()
        else:
            self.report_finish("FAILED")

We subclass “ExternalProcessPlugin“ and use “self.spawn“ to call the Go command-line program. If your program speaks JSON, it is easy to parse the output (and this program is written to mirror the Python code, so the Go program actually outputs the standard issue format that Minion is expecting to use). It would be awesome if other scan tools and Minion can agree on a common report scheme because then writing plugins and exporting to other tools become trivial and possible.

Grow the Ecosystem!

Here is a list of plugins we have developed so far:

  • basic plugins
  • minion-zap-plugin
  • minion-skipfish-plugin
  • minion-setcookie-plugin
  • minion-nmap-plugin
  • minion-ssl-plugin
  • minion-breach-plugin

Also, a community member has been writing a minion-arachni-plugin for Arachni.

But that’s not enough. We want to make security reviews agile by allowing developers to create and to use different plugins to keep their application secured from common vulnerabilities on their own. To achieve this goal, we must build an ecosystem with a great number of useful plugins. If you haven’t heard, Minion IS open source. Fork it on https://github.com/mozilla/minion and help us grow. Minion can become big. Let us know your ideas and talk to us to get you started. Personally, I feel Minion has the potential to grow into something big like OpenStack and Docker.

Anyway, you can reach us via

Although my internship is coming to an end this week, I will continue to contribute on Github like the rest of our Minion developers always do. I want to thank all of you awesome Mozillians and Minion users for the support and guidance over the past 12 weeks. In particular, I want to thank my Web Security Automation team:

  • Stefan Arentz for being Hacker’s Best Mentor and leading Minion development
  • Yvan Boily for providing resources for Minion development
  • Simon Bennetts for mentoring me on improving minion-zap-plugin
  • Mark Goodwin for his initial security review and support

and a big thanks to Stephen Donner (Mozilla Web QA manager) for his commitment to use Minion.

Plug-n-Hack

Simon Bennetts

Plug-n-Hack Overview

Plug-n-Hack (PnH) is a proposed standard from the Mozilla security team for defining how security tools can interact with browsers in a more useful and usable way.

Security researchers commonly use security tools in conjunction with browsers, but until now direct integration has required writing platform and browser specific extensions.

Configuring a browser to work with a security tool can be a non-trivial process, and this can discourage people with less experience from using such tools. This can include application developers and testers, exactly the sort of people we would like to use these tools more!

For example, to configure a browser to use an intercepting proxy that can handle HTTPS traffic, the user must typically:

  • Configure their browser to proxy via the tool
  • Configure the tool to proxy via their corporate proxy
  • Import the tool’s SSL certificate into their browser

If any of these steps are carried out incorrectly then the browser will typically fail to connect to any website – debugging such problems can be frustrating and time-consuming.

Without integration between security tools and browsers, a user must often switch between the tool and their browser several times to perform a simple task, such as intercepting an HTTP(S) request.

PnH allows security tools to declare the functionality that they support which is suitable for invoking directly from the browser.

A browser that supports PnH can then allow the user to invoke such functionality without having to switch to and from the tool.

While some of the PnH capabilities do have a fixed meaning, particularly around proxy configuration, most of the capabilities are completely generic, allowing tools to expose whatever functionality they want.

Implementing the above features in Firefox and the tools that we work on and support gives our team an advantage, however we believe that opening up such capabilities to all browsers and all security tools is much more useful for security researchers and application developers and testers.

As a result we have designed and developed the PnH protocol to be both browser and tool independent. The current protocol and Firefox implementation are released under the Mozilla Public License 2.0 which means it can be incorporated in commercial tools without charge.

Phase 1

PnH phase 1 allows easier integration and defines how security tools can advertise their capabilities to browsers.

To support PnH-1 security tools provide a manifest over HTTP(S) which defines the capabilities that the browser can make use of.

It is up to the tool authors to decide how the URL of the manifest is publicised.

An example manifest (for OWASP ZAP) is:

{
  "toolName":"OWASP ZAP",
  "protocolVersion":"0.2",
  "features":{
    "proxy":{
      "PAC":"http://localhost:8080/proxy.pac",
      "CACert":"http://localhost:8080/OTHER/core/other/rootcert/"
    },
    "commands":{
      "prefix":"zap",
      "manifest":"http://localhost:8080/OTHER/mitm/other/service/"
    }
  }
}

The top level manifest includes optional links to a proxy PAC and a root CA certificate.

It also optionally links to another manifest which describes the commands the browser can invoke.

An example commands manifest (for OWASP ZAP) is: https://code.google.com/p/zap-extensions/source/browse/branches/beta/src/org/zaproxy/zap/extension/plugnhack/resource/service.json

In Firefox the tool commands will be made available via the Developer Toolbar (GCLI) https://developer.mozilla.org/en-US/docs/Tools/GCLI

A example of how the ZAP commands are currently displayed is:

Note that user specified parameters can be specified for commands, which can either be free text, a static pull down list of options or a dynamic list of options obtained from the tool on demand.

So if you select the “zap scan” command then you will be prompted to select a site from the list of sites currently known to ZAP.

PnH does not specify how tool commands should be displayed, so other browsers are free to display them in different ways.

Phase 2

The next phase of PnH is still being planned but is intended to allow browsers to advertise their capabilities to security tools.

This will allow the tools to obtain information directly from the browser, and even use the browser as an extension of the tool.

If you are interested in working on this aspect then please get in touch.

Get involved

While this project has been started by the Mozilla Security Team and has been validated with Firefox and OWASP ZAP, this is an open project and we welcome involvement from anyone, especially people working on other browsers and security tools.

If you would like to add PnH support to a browser or tool, or even get involved in onward PnH development,  then please get in touch and we will give you whatever assistance we can.

Tools supporting PnH

  • OWASP ZAP 2.2.0: via MITM-conf add-on
    Source code available from: zap-extensions
  • Burp Suite: support coming soon

 

Introducing FuzzDB

amuntner

4

FuzzDB is an open source database of attack patterns, predictable resource names,  regex patterns for identifying interesting server responses, and documentation resources. It’s most often used testing the security of web applications but can be useful for many other things. FuzzDB started off as years of my own personal documentation and research notes and gradually evolved into its current form.

This is the first of a series of blog posts about FuzzDB. It discusses:

  • The problem that led to the creation of FuzzDB
  • What kinds of things are in FuzzDB
  • The different ways in which FuzzDB could be used
  • The future of FuzzDB

FuzzDB, is hosted at Google Code: https://code.google.com/p/fuzzdb/

Thinking About Test Cases

A lot of attention has been paid to identifying attackable surface areas, but less to the development of attack pattern libraries. When we dynamically test web applications for security vulnerabilities, how good are the test cases we’re using?

Commercial web scanning tool vendors put significant research effort into this problem, but the product of this research is considered intellectual property and locked up inside the application. As users, in order to learn what kinds of test cases are being generated we would need to painstakingly record and analyze its traffic. At the time I initially released FuzzDB, most open source web fault injection tools had sets of test cases which were woefully incomplete and inadequate. There are too many permutations of symbols and encodings used in web protocols for anyone to reliably and repeatably recall all of them. As for the commercial tools, how complete are their sets of test cases, anyway? It’s not always easy to tell. What were they actually testing for? These tools aren’t just test case lists, they’re lists wrapped in complex sets of rules that determine which test cases to use when  and where. After considering these details, I had some doubts about the effectiveness of the typical application testing process.

My thoughts turned to increasing the speed and accuracy with which I could find certain classes of vulnerabilities during assessments. I began collecting, categorizing, and using lists of attack strings and of common file and directory names. Eventually I organized them into what is now FuzzDB and made it freely available under an Open Source license, the Creative Commons Attribution license.

As with any tool, an individual with malicious intent could potentially use FuzzDB in bad ways. However, I believe that it’s better to provide this information for the security of all. More importantly, if developers and testers have access to a good set of test cases, software will be released that has already passed this list of test cases.

That’s my ultimate goal for FuzzDB: for it to become obsolete as an attack tool because the applications become more secure. When applications and frameworks are inoculated against its patterns through testing and secure coding techniques, bad actors will no longer find the patterns in FuzzDB to be useful.

What’s in FuzzDB?

Predictable Resource Locations - Because there are a small number of popular server OS and infrastructure application packaging systems, resources such as logfiles and administrative directories are typically located in a small number of predictable locations. FuzzDB contains a comprehensive database of these, categorized by OS platform, web server, and  application. The intent is for a tester to use these lists to be able to make educated rather than brute-force guesses, significantly increasing the likelihood of successfully forcible browsing interesting and vulnerable resources. Also, they’re appropriate to be used in creating automated scanners as well as IDS/IPS signatures.

Attack Patterns – The attack pattern test-case sets are categorized by platform, language, and attack type. These are malicious and malformed inputs known to cause information leakage and exploitation. FuzzDB contains comprehensive lists of attack payloads known to cause issues like OS command injection, directory listings, directory traversals, source exposure, file upload bypass, authentication bypass, http header crlf injections, and more.

When I say “malicious inputs,” I mean it. Downloading the project may cause antivirus alerts or trigger pattern-based malicious code sensors. While FuzzDB is itself nothing but a collection of text files that are harmless on their own, some of the patterns included in the files have been used extensively in worms, malware, and other exploits.

Response Analysis - Since system responses also contain predictable strings, FuzzDB contains a set of regex pattern dictionaries such as interesting error messages to aid detection software security defects, lists of common Session ID cookie names, regex for numerous Personally Identifiable Information, and more.

Documentation – Helpful documentation and cheatsheets sourced from around the web that are relevant to the payload categories are provided.

Other useful stuff – Webshells, common password and username lists, and some handy wordlists.

You can browse it’s contents using Google Code’s Source browser.

What can FuzzDB be used for?

  • Web application penetration testing using popular penetration testing tools like OWASP Zap or Burp Suite
  • A standard ZAP Intercepting Proxy add-on
  • Building new automated scanners and automation-assisted manual penetration test tools
  • Testing network services that use something other than HTTP semantics
  • As malicious inputs for testing GUI or command-line software
  • Using the patterns to make your open source or commercially licensed application better
  • Identifying interesting responses to your probes. Here is a screenshot illustrating how this looks in Burp Suite
  • Testing your IDS or IPS by using these test cases to “attack” your web server
  • Testing during a bake-off of web security product vendors
  • Testing a new custom web server or other network service for vulnerability to the patterns that have worked on one or more other platforms in the past
  • Building intrusion identification and response systems
  • Winning app security Capture the Flag competitions
  • As a learning tool for better understanding various different malicious byte combinations which can cause the same vulnerability

If you’re using FuzzDB in a novel way, I’d love to hear about it!

The Future of FuzzDB

There is still a lot of work to be done to improve FuzzDB. My plan for the upcoming year includes:

  • Respond to the outstanding bugs
  • Come up with a consistent naming structure (this is actually one of the bugs)
  • Write more documentation, such as these blog posts
  • Update the Discovery files, they’re still very useful, but a few years old.
  • Improve some of the Attack payload categories
  • Help it work better with OWASP Zap and Minion

In addition, FuzzDB will move into a wiki that will allow discussion of the contents and permit collaboration on new items.
If you’re interested in helping in any of these areas or have suggestions such as a consistent directory and name format for FuzzDB or have more  fuzz files to send, I’d love to hear from you.

Investigating Security Vulnerability Report

mcoates

8

Update – August 5, 2013

Issue
Mozilla was notified on August 4, 2013 of a potential security vulnerability with Firefox 17 (current general release is Firefox 22). Upon investigation we confirmed the vulnerability and determined the root of the issue was related to MFSA 2013-53. This vulnerability was fixed in Firefox versions 17.0.7 and 22, which were released on June 25, 2013.

Impact
Users who are on the latest version of Firefox (version 22) or Firefox ESR (version 17.0.7) are not at risk. If a user is running an outdated version of Firefox, then this vulnerability could be used by an attacker to execute malicious software on a victim’s machine. Mozilla has been alerted that this issue is being actively exploited in the wild and urges all users to make sure their Firefox is up to date.

Status
This vulnerability was fixed in Firefox versions 17.0.7 and 22, which were released on June 25, 2013. Firefox users should follow these instructions to confirm they are running the latest version of Firefox (currently version 22 and 17.0.7 for ESR) which contains the fixes for this vulnerability.

Original Post

Mozilla has been notified of a potential security vulnerability in Firefox 17. Firefox 17 is currently the extended support release version.

We are actively investigating this information and we will provide additional information when it becomes available.

 

Michael Coates
Director of Security Assurance