{"id":125,"date":"2020-05-12T04:20:18","date_gmt":"2020-05-12T11:20:18","guid":{"rendered":"https:\/\/blog.mozilla.org\/attack-and-defense\/?p=125"},"modified":"2020-06-16T07:24:13","modified_gmt":"2020-06-16T14:24:13","slug":"fuzzing-firefox-with-webidl","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/attack-and-defense\/2020\/05\/12\/fuzzing-firefox-with-webidl\/","title":{"rendered":"Fuzzing Firefox with WebIDL"},"content":{"rendered":"
<\/div>\n

TL;DR, An Introduction<\/h2>\n

This post originally appeared on Mozilla Hacks.<\/em><\/small><\/a><\/p>\n

Fuzzing<\/em>, or fuzz testing, is an automated approach for testing the safety and stability of software. It\u2019s typically performed by supplying specially crafted inputs to identify unexpected or even dangerous behavior. If you\u2019re unfamiliar with the basics of fuzzing, you can find lots more information in the Firefox Fuzzing Docs<\/a> and the Fuzzing Book<\/a>.<\/p>\n

For the past 3 years, the Firefox fuzzing team has been developing a new fuzzer to help identify security vulnerabilities in the implementation of WebAPIs in Firefox. This fuzzer, which we\u2019re calling Domino, leverages the WebAPIs’ own WebIDL<\/a> definitions as a fuzzing grammar. Our approach has led to the identification of over 850 bugs<\/a>. 116 of those bugs have received a security rating. In this post, I\u2019d like to discuss some of Domino\u2019s key features and how they differ from our previous WebAPI fuzzing efforts.<\/p>\n

Fuzzing Basics<\/h2>\n

Before we begin discussing what Domino is and how it works, we first need to discuss the types of fuzzing techniques available to us today.<\/p>\n

Types of Fuzzers<\/h3>\n

Fuzzers are typically classified as either blackbox, greybox, or whitebox. These designations are based upon the level of communication between the fuzzer and the target application. The two most common types are blackbox and greybox fuzzers.<\/p>\n

Blackbox Fuzzing<\/h4>\n

Blackbox fuzzing submits data to the target application with essentially no knowledge of how that data affects the target. Because of this restriction, the effectiveness of a blackbox fuzzer is based entirely on the fitness of the generated data.<\/p>\n

Blackbox fuzzing is often used for large, non-deterministic applications or those which process highly structured data.<\/p>\n

Whitebox Fuzzing<\/h4>\n

Whitebox fuzzing enables direct correlation between the fuzzer and the target application in order to generate data that satisfies the application\u2019s \u201crequirements\u201d. This typically involves the use of theorem solvers to evaluate branch conditions and generate data to intentionally exercise all branches. In doing so, the fuzzer can test hard-to-reach branches that might never be tested by blackbox or greybox fuzzers.<\/p>\n

The downside of this type of fuzzing\u2014it is computationally expensive. Large applications with complex branching may require a significant amount of time to solve. This greatly reduces the number of inputs tested. Outside of academic exercises, whitebox fuzzing is often not feasible for real-world applications.<\/p>\n

Greybox Fuzzing<\/h4>\n

Greybox fuzzing has emerged as one of the most popular and effective fuzzing techniques. These fuzzers implement a feedback mechanism, typically via instrumentation, to inform decisions on what data to generate in the future. Inputs which appear to cover more code are reused as the basis for later tests. Inputs which decrease coverage are discarded.<\/p>\n

This method is incredibly popular due to its speed and efficiency in reaching obscure code paths. However, not all targets are good candidates for greybox fuzzing. Greybox fuzzing typically works best with smaller, deterministic targets that can process a large number of inputs quickly<\/a> (several hundred a second).<\/p>\n

We often use these types of fuzzers to test individual components within Firefox such as media parsers. If you\u2019re interested in learning how to leverage these fuzzers to test your code, take a look at the Fuzzing Interface documentation<\/i> here<\/i><\/a>.<\/i><\/p><\/blockquote>\n

Unfortunately, we are somewhat limited in the techniques that we can use when fuzzing WebAPIs. The browser by nature is non-deterministic and the input is highly structured. Additionally, the process of starting the browser, executing tests, and monitoring for faults is slow (several seconds to minutes per test). With these limitations, blackbox fuzzing is the most appropriate solution.<\/p>\n

However, since the inputs expected by these APIs are highly structured, we need to ensure that our fuzzer generates data that is considered valid.<\/p>\n

Grammar-Based Fuzzing<\/h3>\n

Grammar-based fuzzing is a fuzzing technique that uses a formal language grammar to define the structure of the data to be generated. These grammars are typically represented in plain-text and use a combination of symbols and constants to represent the data. The fuzzer can then parse the grammar and use it to generate fuzzed output.<\/p>\n

\"A<\/a><\/p>\n

The examples here demonstrate two simplified grammar excerpts from the Domato<\/a> and Dharma<\/a> fuzzers. These grammars describe the process of creating an HTMLCanvasElement<\/a><\/code> and manipulating its properties and operations.<\/p>\n

Issues with Traditional Grammars<\/h4>\n

Unfortunately, the level of effort required to develop a grammar is directly proportional to the size and complexity of the data you\u2019re attempting to represent. This is the biggest downside of grammar-based fuzzing. For reference, WebAPIs in Firefox expose over 730 interfaces with approximately 6300 members. Keep in mind, this number does not account for other required data structures like callbacks, enums<\/a>, or dictionaries, to name a few. Creating a grammar to describe these APIs accurately would be a huge undertaking; not to mention error-prone and difficult to maintain.<\/p>\n

To more effectively fuzz these APIs, we wanted to avoid as much manual grammar development as possible.<\/p>\n

 <\/p>\n

WebIDL as a Fuzzing Grammar<\/h2>\n
typedef (BufferSource or Blob or USVString) BlobPart;\r\n\r\n[Exposed=(Window,Worker)]\r\ninterface Blob {\r\n [Throws]\r\n constructor(optional sequence blobParts,\r\n             optional BlobPropertyBag options = {});\r\n\r\n [GetterThrows]\r\n readonly attribute unsigned long long size;\r\n readonly attribute DOMString type;\r\n\r\n [Throws]\r\n Blob slice(optional [Clamp] long long start,\r\n            optional [Clamp] long long end,\r\n            optional DOMString contentType);\r\n [NewObject, Throws] ReadableStream stream();\r\n [NewObject] Promise text();\r\n [NewObject] Promise arrayBuffer();\r\n\r\n};\r\n\r\nenum EndingType { \"transparent\", \"native\" };\r\n\r\ndictionary BlobPropertyBag {\r\n DOMString type = \"\";\r\n EndingType endings = \"transparent\";\r\n};<\/code><\/pre>\n
A simplified example of the Blob WebIDL definition<\/i><\/p>\n
WebIDL<\/a>, is an interface description language<\/a> (IDL) for describing the APIs implemented by browsers. It lists the interfaces, members, and values exposed by those APIs as well as the syntax.<\/p>\n
The WebIDL definitions are well known among the browser fuzzing community because of the wealth of information contained within them. Previous work has been done in this area to extract the data from these IDLs for use as a fuzzing grammar, namely the WADI fuzzer from Sensepost<\/a>. However, in each example we investigated, we found that the information from these definitions was extracted and re-implemented using the fuzzer\u2019s native grammar syntax. This approach still requires a significant amount of manual effort. And further, the fuzzing grammars’ syntax make it difficult, if not impossible in some instances, to describe behaviors specific to WebAPIs.<\/p>\n
Based on these issues, we decided to use the WebIDL definitions directly, rather than converting them to an existing fuzzing grammar syntax. This approach provides us with a number of benefits.<\/p>\n
Standardized Grammar<\/h3>\n
First and foremost, the WebIDL specification defines a standardized grammar to which these definitions must adhere. This lets us leverage existing tools, such as WebIDL2.js<\/a>, for parsing the raw WebIDL definitions and converting them into an abstract syntax tree<\/a> (AST). Then this AST can be interpreted by the fuzzer to generate testcases.<\/p>\n
Simplified Grammar Development<\/h3>\n
Second, the WebIDL defines the structure and behavior of the APIs we intend to target. Thus, we significantly reduce the amount of required rule development. In contrast, if we were to describe these APIs using one of the previously mentioned grammars, we would have to create individual rules for each interface, member, and value defined by the API.<\/p>\n
ECMAScript Extended Attributes<\/h3>\n
Unlike traditional grammars, which only define the structure of data, the WebIDL specification provides additional information regarding the interface\u2019s behavior via ECMAScript extended attributes. Extended attributes can describe a variety of behaviors including:<\/p>\n