The Fuzzing Project

I create quarterly reports for the Core Infrastructure Initiative about the progress of the Fuzzing Project. The September 2015 report can now be downloaded from their webpage.

It includes some notes about work I've been doing on creating a full Gentoo Linux system built with Address Sanitizer, some information about Undefined Behavior Sanitizer and Kernel Address Sanitizer, fuzzing of filesystem tools and about the recent BIND Denial of Service vulnerability.

Address Sanitizer is a remarkable feature that is part of the compilers gcc and clang. I make heavy use of it and it can uncover many memory access bugs that would otherwise be hard to find.

What may be lesser known is that there is also the possibility to use Address Sanitizer for the Linux Kernel. It is available as an option since version 4.0.

I recently tried for the first time to boot a kernel with Kernel Addres Sanitizer (KASAN). There are a few things to consider. Appart from the option CONFIG_KASAN one should also set CONFIG_STACKTRACE. There are two variants how KASAN can be enabled, CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. The latter seems to be preferrable, it's faster, but it creates a larger kernel binary and requires a relatively recent gcc version (5.0 or later).

To my surprise just booting a kernel with KASAN already showed a number of warnings about out of bounds errors. Unlike normal ASAN these errors don't cause a crash (that would be quite inconvenient in the kernel). Instead they just print a warning in the dmesg log. Also by itself the kernel is not able to print an error message with line numbers. One needs to pass the output to a script that is available here.

Most of the warnings came from a preprocessor macro in the Intel GPU driver. I spend some time fixing the issue and had a working patch ready. Then I found out that it was already fixed in the current git code... (Remember: Always check if code is already fixed in git if you try to fix a bug.)

Anyway, having fixed that issue silenced most of the warnings, but a few remained. I could track them down to a wrong use of a counter variable in a nested loop. This loop was supposed to check the correct sorting of a table of commands. However it turned out that the tables weren't properly sorted and the fix made the kernel unbootable. After some discussions with the Intel driver developers I was able to finally fix the issue with two patches which have just been merged into the main kernel tree (they are in 4.3-rc1).

Given that just booting the kernel with KASAN enabled was enough to uncover some bugs indicates that not enough people have tested it yet. I also tried some kernel fuzzing tools with KASAN enabled (perf_fuzzer and trinity) and tried to mount a couple of corrupted filesystem images generated by american fuzzy lop. That didn't turn up any further bugs.

The latest update for the BIND DNS server fixes a bug that could be used to crash a DNS server that is verifying DNSSEC records. Parsing a malformed DNSSEC key can lead to an assert in the file buffer.c.

Usually DoS-issues are considered relatively minor. In this case however I consider the impact relatively severe as it is quite easy to trigger crashes of a large number of DNS servers. It is often easy to force a system to resolve a certain domain name, e. g. through a website (clients) or an e-mail (servers). Although DNSSEC is not widely deployed a lot of DNS resolvers have DNSSEC validation enabled.

This issue was found with american fuzzy lop. BIND ships several command line tools that parse DNSSEC keys, e. g. dnssec-importkey, dnssec-dsfromkey or dnssec-revoke that can trigger this bug. Given that quite recently another vulnerability in BIND was also found with american fuzzy lop it is quite surprising to me that this issue wasn't found earlier. There was almost nothing special in fuzzing the BIND tools, the only thing to consider was that they expect keys in a certain file name scheme (they need to start with a K). afl-fuzz can guarantee certain filenames with the -f parameter. This tells us that even in highly critical software like BIND one can sometimes still find vulnerabilities with afl easily.

This issue was reported to ISC (the company developing BIND) on August 1st. After exchanging a couple of mails the BIND developers provided me a test patch on August 5th. The releases with the fixes were released on September 2nd. The fix is contained in the releases 9.0.7-P3 and 9.10.2-P4. These releases also fix another security issue (CVE-2015-5986). If you're running an affected BIND version you should update immediately.

I want to thank Florian Weimer from Red Hat who confirmed that this issue is remotely exploitable through a DNSSEC zone.

To give people some time to patch their servers I will wait a few days until I publish the proof of concept crasher. This should however be no excuse not to update your servers. It is quite possible that others will reproduce this work and create a working exploit very soon.

ISC advisory: Parsing malformed keys may cause BIND to exit due to a failed assertion in buffer.c
CVE-2015-5986

Update (2015-09-11):

The proof of concept is now public. This can be tested with several command line tools from bind, e. g. with dnssec-dsfromkey.

The key record looks like this:
0 DNSKEY 0 0 2 ADN00000000000000000000000000000000000000000000000000000000000000000000AC000

I have personally not tested to create a live DNSSEC record crashing BIND installations (I don't use DNSSEC myself). It should be enough to add that as a DNSKEY record in your setup and try to resolve the domain with a DNSSEC-validating, vulnerable BIND resolver.

Update 2 (2015-09-15):

I was asked whether I can also provide a proof of concept for CVE-2015-5986, which was fixed in the same release. This issue wasn't discovered by me, but I was easily able to fuzz that one as well, here is the proof of concept file. It can be tested with named-checkzone -f raw -o - a [infile].