Fuzz testing (automated, random testing) is an important part of nearly every application security life cycle. While there are a lot of tools, frameworks and harnesses available for regular desktop platforms/operating systems, there’s still a lot missing in the mobile sector which is becoming increasingly important.
Basic Requirements for Mobile
The principles of fuzzing are no different on mobile when comparing to desktop platforms, it’s just that the different steps of fuzzing are more complex to implement. To understand the problems we need to solve, we first have to look at these steps from a more general perspective.
The following diagram roughly describes how many fuzzers work:
- We need to be able to start the target program obviously.
- Supply tests to the program, which can either happen by testing from inside the program, or by creating tests externally and sending them to the program.
- While a test is executed, we need to monitor the program for responsiveness and crashes.
- If the program goes non-responsive for a certain amount of time, we can assume that it’s hanging and we terminate the program and go back to the first step.
- If we crash, then we would of course want to triage the crash and e.g. determine if it’s a known problem.
- Based on the triage, we need to save the tests we executed so far in order to be able to reproduce the problem later. Note that based on the correctness assumptions, we might also consider hangs as interesting and process them like crashes (e.g. if our program is not supposed to hang under any circumstances).
Let’s try to apply this information to Mobile: Assuming our target program is running on a mobile device (e.g. tablet, smart phone), we first need a way to communicate with the device to perform basic tasks like start/stop the program and monitor if it’s still running. For Android, the Android Debugging Bridge (ADB), offers the necessary functionality. Using the
adb command line tool, we can perform various tasks on the device, including running all sorts of shell commands which should give us enough flexibility to accomplish these steps.
However, we do not need to use the adb command line tool directly: Mozilla has a Python-based abstraction called DeviceManager which supports certain tasks on mobile that are implemented using ADB. Using this implementation, we don’t have to care about low-level shell implementation anymore but can instead use a high-level interface.
Communication with the Program
Once we have such a connection we can use it to implement responsiveness protocols and logging as it is desired. Optionally we can use it to send tests to the browser to have them executed there.
In general, crash triage can be done reasonably well once we have a crash stack trace and an optional assertion message. On the desktop, the OS usually provides some form of crash dump, like core dumps on Linux. On Android, this isn’t trivial to achieve. While it’s certainly possible to get core dumps to work on a rooted Android device, it’s impossible without rooting (to my knowledge). But at least for Firefox, we don’t need core dumps, we can use the crash reporter built into Firefox instead. With the right settings, the Firefox crash reporter will create a minidump when it crashes, but not submit it or show a submit screen. Instead it will just exit after having written the dump, which is perfectly suitable for our automated testing.
Once the fuzzing harness detects a crash, we only have to check the minidump directory on the device for any crash dumps, then pull the dump off the device and run a tool like minidump_stackwalker on it to get a crash stack trace.
Proof of Concept
For now, the implementation only comes with a “Hello world” fuzzer, that includes the required Websockets communication code but no real fuzzing logic itself. Instead it has a nice little demo that will spin around a
<div> tag using CSS transformations. It is up to you to implement any desired “real” testing there. As previously mentioned, we also plan to release certain tools in the future: Jesse Ruderman, author of jsfunfuzz, domfuzz and other fuzzing tools we use at Mozilla, has announced that he will very likely make his tools public in the near future. Some of these tools are compatible with the harness presented here.
Special thanks goes to the Mozilla Automations Team (ateam), especially to Joel Maher, Geoffrey Brown, William Lachance and Clint Talbert for helping me with necessary changes around DeviceManagerADB.
– Christian Holler