To facilitate communication between Firefox and Nyx, we use a custom agent<\/a>, essentially glue-code that is preloaded into Firefox. This code handles the low-level communication with Nyx and is also responsible for providing the trace buffer (for coverage measurements) to the AFL++ runtime linked to Firefox as well as passing through fuzzing data from AFL++. Both of these tasks are more complex in this configuration as AFL++ is not directly launching and communicating with the target binary. The agent further exposes a clean interface<\/a> to Firefox that can be used to implement the actual fuzzer in Firefox itself without having to worry about the low-level details.<\/p>\n Technology stack for snapshot fuzzing at a glance.<\/p><\/div>\n On top of this interface, we have implemented multiple IPC fuzzing targets<\/a>, the simplest one being IPC_SingleMessage<\/a>, which we will look at in more detail now.<\/p>\n Modifying a single IPC message in transit is one of the rudimentary approaches\u00a0 for IPC fuzzing in general. It is especially useful if the message type being targeted is in itself complex (lots of data contained in a single message rather than a complex interface being composed of a large number of simpler messages).<\/p>\n For this purpose, we intercept messages in the parent process on the target thread before they are dispatched<\/a> to the generated IPC code that ultimately calls the IPC method. Most of the logic is then contained in IPCFuzzController::replaceIPCMessage which primarily does either of these two things:<\/p>\n Once the fuzzed message is dispatched (most commonly to a different thread), we face an important challenge\u00a0 of multi-threaded snapshot fuzzing: synchronization. Coverage-guided fuzzing generally operates under the assumption that we know when our fuzzing data has been processed. Depending on the fuzzing target, it can be fairly difficult to tell when we are \u201cdone\u201d but in our case, because we are already on the target thread that is running the actual IPC method. So unless that method again performs an asynchronous dispatch, we can just wait for the dispatch to return and we do so at the end of DispatchMessage() where we call back into IPCFuzzController<\/a> to release (revert back to the snapshot).<\/p>\n By combining this target with a CI test\u00b9, we are now able to find implementation flaws like for example a vulnerability in the accessibility code<\/a> that involved the ShowEvent<\/a> message. This message contains an array of serialized AccessibleData, making this message type a good target for single message fuzzing.<\/p>\n Code coverage is probably the most important metric for long-term fuzzing campaigns as it highlights potential shortcomings of the fuzzing. While for most fuzzing, it is rather straightforward to generate code coverage, doing so in snapshot fuzzing is less trivial. Traditional source code coverage provided by tools like gcov<\/a> which find usage with other fuzzing, aren\u2019t easily deployable because the data would have to be pulled out of the VM on every iteration so it can be saved before the snapshot revert resets the data. Doing so would make the process of obtaining code coverage unfeasibly slow.<\/p>\n Instead, we decided to build our own code coverage measurement on top of the existing instrumentation. For this purpose, we added a new AFL++ instrumentation type<\/a> that instruments all basic blocks and then creates a second, permanent trace buffer in AFL++<\/a> that accumulates the coverage of the regular trace buffer. Finally, we create a third buffer called the pcmap<\/i> which maps every entry in the trace buffer to an address in the binary that can later be resolved to a source code location using debug information. As this information is contained<\/a> in the AFL++ runtime, we need to obtain it within our custom Nyx agent and write it out to the host<\/a>. The same holds for module information<\/i><\/a> that denotes at which addresses Firefox modules were loaded. By combining these three sources of information, we can map the progress of Nyx fuzzing onto actual source code. We also built additional tooling<\/a> to turn this basic block coverage into line-based coverage using information from a gcov build\u00b2. As a result, we can generate metrics like percentage of code covered to evaluate the overall effectiveness of snapshot-based fuzzing.<\/p>\n While snapshot fuzzing is a rather complex technology with many moving parts, it allows us to effectively stress code regions\u00a0 of the browser that would otherwise\u00a0 remain beyond the\u00a0 capabilities\u00a0 of traditional fuzzing techniques,\u00a0 but are critical for providing adequate security guarantees. We are happy to report that this new fuzzing technology is becoming the norm and is now an essential part of our security testing strategy. We would like to thank the authors of Nyx and AFL++ for making this technology available and hope that our combined efforts will help others to adapt snapshot fuzzing for their projects.<\/p>\n \u00b9 Firefox runs many C<\/b>ontinuous I<\/b>ntegration tests to ensure every functionality of the browser is automatically tested.<\/small><\/p>\n \u00b2 Unfortunately, gcov and debug information deviate in some cases, so the result is not a 100% accurate mapping yet and can\u2019t be seamlessly merged with other gcov data. This could likely be improved using LLVM annotations for additional basic block information.<\/small><\/p>\n","protected":false},"excerpt":{"rendered":" Process separation remains one of the most important parts of the Firefox security model and securing our IPC (Inter-Process Communication) interfaces is crucial to keep privileges in the different processes … Read more<\/a><\/p>\n","protected":false},"author":409,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[449530,449525],"tags":[542,465201,465200,465203,465202,465199],"coauthors":[45537],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/attack-and-defense\/wp-json\/wp\/v2\/posts\/268"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/attack-and-defense\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/attack-and-defense\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/attack-and-defense\/wp-json\/wp\/v2\/users\/409"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/attack-and-defense\/wp-json\/wp\/v2\/comments?post=268"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/attack-and-defense\/wp-json\/wp\/v2\/posts\/268\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/attack-and-defense\/wp-json\/wp\/v2\/media?parent=268"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/attack-and-defense\/wp-json\/wp\/v2\/categories?post=268"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/attack-and-defense\/wp-json\/wp\/v2\/tags?post=268"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mozilla.org\/attack-and-defense\/wp-json\/wp\/v2\/coauthors?post=268"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"The](\"http:\/\/blog.mozilla.org\/attack-and-defense\/files\/2024\/06\/snapshot-stack.png\")
Fuzzing a single IPC message<\/h2>\n
\n
Measuring Code Coverage in Snapshot Fuzzing<\/h2>\n
Conclusion<\/h2>\n
\n