Skip to content

Multiple vulnerabilities in RPM – and a rant

Last year in November I decided that it might be a good idea to fuzz the parsers of package management tools in Linux distributions. I quickly found a couple of issues in DPKG and RPM. For DPKG the process went very smooth. I reported them to Debian's security team, eight days later fixes and security advisories were published by both Debian and Ubuntu, the main distributions using DPKG. For RPM the process was a bit more difficult.

If you want to report a bug to RPM you first may wonder where to report it. The RPM webpage [1] is a trac installation which has its own bug tracker. However if you try to register an account there you'll get forwarded to an HTTPS site with an expired certificate that doesn't match the domain name. In case you are brave and tell your browser to ignore all warnings you'll be greeted by a broken-looking trac without any CSS. Should you proceed and create an account you will learn that this doesn't help you, because in order to be allowed to report a bug you first have to ask on the mailing list or in the IRC channel for permission [2]. That's probably the point where many well-meaning bug reporters give up.

Okay, but RPM stands for “Red Hat package manager”, so maybe Red Hat feels responsible. So I reported three bugs with sample files triggering them to the Red Hat security team on November 20th. The answer was – to put it mildly – a bit dissatisfying. I'll just fully quote it: “Thanks for the report. We also received about 30+ crash reports in RPM from
a different reporter recently so processing all of them (yours and the
others) will take quite a bit of time. We simply don't have the resources
to spend hours upon hours analyzing all crash reports.”

Okay, so I wasn't the only one fuzzing RPM and the maybe bugs will be fixed some day. I waited. In the meantime I got contacted by another person who also had tried to report fuzzing bugs in RPM and who has made similar experiences (maybe the same person who reported the 30+ crashers, I don't know).

In February I decided to ask what the state of things is. I also gave them a 30 day period until I'd publish the bugs (I know that it's now long past that, I shouldn't have let this issue wait so long). I ended up having a call with a Red Hat security team member and exchanged a couple of mails. I learned that RPM has a Github repository [3], which contains fixes for some (but not all) of the issues I reported, however that's nowhere referenced on its webpage. I then fuzzed the current RPM git code again and found two more issues I also reported to the Red Hat security team.

Status today is that the latest release of RPM on its webpage – 4.12.0.1 - is from July 2015, so all of the bugs still affect this release. However it seems there is an unofficial 4.13 release that's nowhere to be found on the RPM webpage, but Red Hat is using it together with some fixes [4]. And the Github repository says the latest release is 4.12.0, so according to three different sources three different versions are the current one (4.12.0, 4.12.0.1, 4.13).

One of the bugs – a stack overflow (write) - is still present in the latest code on Github.

Commend and Conclusion

This blog post probably reads like a big rant about how unprofessional Red Hat is in handling potential security issues. But this is contrary to my usual experience. I often see Red Hat developers being very active in the free software security community and often contributing in a positive way. Quite simply I expect better from Red Hat. This is not some dubious Enterprise vendor where I wouldn't be the least bit surprised of such a reaction.

The development process of RPM seems to be totally chaotic, it's neither clear where one reports bugs nor where one gets the latest code and security bugs don't get fixed within a reasonable time.

There's been some recent events that make me feel especially worried about this: An unknown person has created an entry in the Libarchive issue tracker [5] that points to an anonymous document [6] with a very detailed description of various security weaknesses in the FreeBSD update process (most of them are still unfixed). The most worrying thing about this is however that the anonymous post mentions the existence similar documents affecting multiple Linux distributions. These documents haven't shown up publicly yet and given the unclear nature of this incident it's hard to know whether they ever will become public or exist at all. But this should still be reason enough to have a closer look at potential security vulnerabilities in all pieces of Linux package management systems.

I haven't analyzed the RPM installation process in detail, so I can't say how likely it is that the RPM tool ever sees a malformed input file. It seems downloads happen over HTTP, but the first thing that happens is a signature check. As the signature is part of the RPM file it already needs to be parsed for this. The exact impact of these bugs would require further analysis. But independent of how likely this is I think the parser in such a crucial piece of software should be robust. It should be safe to use the rpm tool to show info about a file on the command line.


[1] http://rpm.org/
[2] http://rpm.org/wiki/ReportingBugs
[3] https://github.com/rpm-software-management/rpm
[4] http://pkgs.fedoraproject.org/cgit/rpms/rpm.git/diff/rpm-4.13.0-rpmtd-out-of-bounds.patch?h=f22&id=165614f3dd42caa188f78b55e7723dad2900b2f4
[5] https://github.com/libarchive/libarchive/issues/743
[6] https://gist.github.com/anonymous/e48209b03f1dd9625a992717e7b89c4f

All bugs were found with the help of american fuzzy lop. Here are the bugs:

Stack Overflow in glob() / rpmglob.c.
Sample file (test with rpm -i [input]):
https://crashes.fuzzing-project.org/rpm-stackoverflow-glob.rpm
Unfixed in the current Git code.

Heap out of bounds read in headerVerifyInfo() / header.c.
Sample file (test with “rpm -i [input]”):
https://crashes.fuzzing-project.org/rpm-heap-oob-read-headerVerifyInfo.rpm
Git commit:
https://github.com/rpm-software-management/rpm/commit/8e847d52c811e9a57239e18672d40f781e0ec48e

Null pointer access / segfault in stringFormat() / formats.c
Sample file (test with “rpm -i [input]”):
https://crashes.fuzzing-project.org/rpm-nullptr-rpmtdFormat.rpm
Git commit:
https://github.com/rpm-software-management/rpm/commit/cddf43a56f19711866371f02f378dc4095b0fadd

Out of bounds read in rpmtdGetNumber() / rpmtd.c
Sample file (test with “rpm -qi -p -- [input]”)
https://crashes.fuzzing-project.org/rpm-heap-oob-read-rpmtdGetNumber.rpm
Git commit:
https://github.com/rpm-software-management/rpm/commit/b722cf86200505b3e3fcbb2095c4ff61f1f5a2ab

Finally one annoying thing to admit: In my original report I included another segfault in headerVerifyInfo() with unclear reasons. However I am now unable to reproduce this one. It may be due to compiler options, different command line parameters or dependencies on my system that have changed. For completeness I'm still providing the sample file:
https://crashes.fuzzing-project.org/rpm-segfault-headerVerifyInfo.rpm
(Ideally the RPM developers should include all those sample files in a test suite they regularly run against an address sanitizer build of RPM.)

Please also note that I expect this list to be incomplete and there are likely more issues that could be uncovered with further fuzzing. I'll test that once all the existing issues are fixed.

Fun with Bignums: Crashing MatrixSSL and more

If you've been following my fuzzing work you will be aware that I've fuzzed various bignum libraries and found several bugs by comparing implementations against each other.

I recently had a look at the MatrixSSL's modular exponentiation function, for reasons I'll explain later. I wrote a wrapper, similar to previous experiments, comparing its result to OpenSSL.

I immediately noted that the pstm_exptmod() function of MatrixSSL has certain limitations that weren't documented. If one tries to calculate a modular exponentiation with the base equal to the modulus (a^b mod a, code) it would return an error. If one tries to calculate a modular exponentiation with the base zero (0^b mod a, code) it would crash with an invalid free operation, potentially leading to memory corruption.

In normal cryptographic operations these values should never appear. But these values are in many situations attacker controlled. One situation is during an RSA key exchange. What happens here is that a client encrypts a random secret with the server's key. However a malicious client could simply send a zero or the key's modulus here. I created a patch against openssl that allows to test this. Both values crash the MatrixSSL server. However the crash seems not to happen in pstm_exptmod(), it hits another bug earlier. In both cases the crash happens due to an invalid memory read in the function pstm_reverse(), which is not prepared for zero-sized inputs and will underflow the len variable.

The crashes have been fixed in 3.8.4, but the pstm_exptmod() function still doesn't accept these inputs. However it no longer crashes with a zero base. It may be possible that these issues can be triggered through other code paths. I haven't tested Diffie Hellman key exchanges, which also allows putting attacker-controlled values into a modular exponentiation.

This is an interesting class of bugs. Bignum functions often aren't designed to handle all inputs and only consider values that make sense in the context of the cryptographic operations. However if they are attacker-controlled this may lead to problems. I just discovered a somewhat similar issue in Nettle. They switched their RSA implementation from GMP's mpz_powm() function to mpz_powm_sec(), which is supposed to be sidechannel resistant. However mpz_powm_sec() is no drop-in replacement. Unlike mpz_pown() it doesn't accept even moduli and crashes with a floating point error. Therefore when trying to use a specifically crafted RSA key with an even modulus this will crash. Fortunately this was discovered before the change made it into a release.

But back to MatrixSSL: Independent of these corner case values that lead to failures I was able to identify an input value that caused a wrong calculation result.

There's a particularly severe risk with calculation errors in the modulo exponentiation when it comes to the RSA algorithm. A common way to speed up the calculation of RSA signatures is an algorithm based on the chinese remainder theorem (CRT) that splits it up into two smaller calculations. However if one of these calculations goes wrong an attacker can learn the private key. Last year Florian Weimer observed that various devices had this error and he could extract their keys. He recently mentioned on the oss-security mailing list that he also observed this in devices using MatrixSSL.

The way the MatrixSSL team "fixed" the miscalculation issue is not really satisfying: They now restrict the input to the pstm_exptmod() function to a set of bit sizes (512, 1024, 1536, 2048, 3072, 4096). My test input had a different bit size, therefore I cannot reproduce the miscalculation any more, but the underlying bug is most likely still there. I've tried to find inputs matching these restrictions and still causing wrong results, but without success yet. Independent of that the restriction means that connections to sites with unusual key sizes or Diffie Hellman moduli will no longer work. While they are not common, there is no rule that RSA keys or Diffie Hellman moduli need to have certain sizes.

Despite the fact that the bug may be still there the CRT attack will probably no longer work. A protection mechanism against that was implemented in version 3.8.3.

I got told by the MatrixSSL developers that their bignum code is based on libtommath. Therefore I also checked if the same bugs appeared there. That wasn't the case. The test input causing wrong results in MatrixSSL were correctly calculated by libtommath and it was also capable of correctly using a zero base or a base equal to the modulus.

Multiple issues in p7zip

I reported several bugs discovered with afl in p7zip, the portable command line version of the 7-Zip archiving tool, one null pointer access causing a segfault and one out of bounds heap read.

At the time I reported the first issues these were already fixed in the main 7-Zip code, but the portable version hadn't been updated yet. Now there is version 16.02, unfortunately there are still issues left.

Null pointer access / segfault
An access to a null pointer in NArchive::N7z::CStreamSwitch::Set can cause p7zip to crash.

Out of bounds read (bug is not publicly visible yet)
An out of bounds heap read access can happen in the function NArchive::N7z::CDecoder::Decode.

Null pointer acctess / segfault (unfixed in 16.02)
A memory allocation isn't checked for a failing allocation. If the file's header indicates a large data size the allocation will fail and subsequently p7zip will try to write to a null pointer. This will likely only lead to a crash and not to further security issues, however this problem seems prevalent in the 7-zip codebase - there are several memory allocations without a check for failures.

p7zip is often used in security appliances to scan inputs, because it supports a large number of archiving file formats. Vulnerabilities in it have played a role in previous attacks on Fireeye appliances. Therefore it's worrying that it's so easy to find crashers in it via fuzzing.

There were also other issues in 7-Zip discovered lately.

Out of bounds read and signed integer overflow in libarchive

I recently wrote about a large number of bugs and potential security issues in libarchive. The release 3.2.0 missed one fix for an out of bounds read in the rar parser. Also I discovered one additional signed integer overflow issue with ubsan. Both issues are now fixed in libarchive 3.2.1.
All issues were discovered with the help of american fuzzy lop.

Out of bounds heap read in RAR parser
Sample rar file
CVE-2015-8934

Signed integer overflow in ISO parser
Sample ISO file
CVE-2016-5844

Also a couple of other security issues in libarchive were found by Cisco.

With the release of version 3.2.1 I consider libarchive to be reasonably robust against fuzzing. I've tested all supported file formats and fuzzed each one with afl/asan for at least one day. Of course that doesn't mean that no security issues are left - but the easy to find ones should be wiped out.

Many invalid memory access issues in libarchive

libarchive version 3.2.0 (released on April 30th) fixed a large number of memory access bugs that I reported to them a while ago. All issues (except the test suite failure) were found with the help of american fuzzy lop and either address sanitizer or undefined behavior sanitizer.

Unclear invalid memory read in CPIO parser
Sample file
CVE-2015-8915

Null pointer access in RAR parser
Sample file
CVE-2015-8916

Null pointer access in CAB parser
Sample file
CVE-2015-8917

Overlapping memcpy in CAB parser
Sample file
CVE-2015-8918

Heap out of bounds read in LHA/LZH parser
Sample file
CVE-2015-8919

Stack out of bounds read in ar parser
Sample file
CVE-2015-8920

Global out of bounds read in mtree parser
Sample file
CVE-2015-8921

Null pointe access in 7z parser
Sample file
CVE-2015-8922

Unclear crashes in ZIP parser
Sample file
CVE-2015-8923

Heap out of bounds read in TAR parser
Sample file
CVE-2015-8924

Unclear invalid memory read in mtree parser
Sample file
CVE-2015-8925

Null pointer access in RAR parser
Sample file
CVE-2015-8926

Heap out of bounds heap read read when reading password for malformed ZIP
Sample file
CVE-2015-8927

Heap out of bounds read in mtree parser
Sample file
CVE-2015-89208

I also reported a couple of lower severity issues (leaks, hangs, undefined behavior issues):

Memory leak in TAR parser
CVE-2015-8929

Endless loop in ISO parser
Sample file
CVE-2015-8930

Undefined behavior / signed integer overflow in mtree parser
CVE-2015-8931

Use after free in test suite

Undefined behavior / invalid shiftleft in TAR parser
Sample file
CVE-2015-8932

Undefined behavior / signed integer overflow in TAR parser
Sample file
CVE-2015-8933

Unfortunately one out of bounds heap read bug in the RAR parser (CVE-2015-8934, sample file) remained unfixed. I hope a fix will find its way into the next version. I was interested in making libarchive more robust because once all issues are fixed it can serve as a safer alternative to many low quality command line tools for various archiving formats.

Various invalid memory accesses in ImageMagick (WPG, DDS, DCM)

Further fuzzing of ImageMagick uncovered some more issues.

An out of bounds memory read in the VerticalFilter() function can be triggered by a malformed DDS file.
Sample file
Git commit / fix
CVE-2016-5687
This was fixed in versions 7.0.1-4 and 6.9.4-3.

Several bugs in the WPG parser could lead to a heap overflow and random invalid memory writes. These bugs only seem to appear when a memory limit is set.
Sample for heap write overflow in SetPixelIndex
Sample for unclear invalid write in ScaleCharToQuantum
Sample for unclear invalid write in SetPixelIndex
Git commit / fix 1
Git commit / fix 2
CVE-2016-5688
These issues were fixed in versions 7.0.1-4 and 6.9.4-3.

Null pointer accesses and unclear segfaults can happen in the DCM parser.
Sample for null pointer access in ReadDCMImage
CVE-2016-5689
Sample for null pointer access in ReadDCMImage (different code)
CVE-2016-5690
Sample for unclear segfault in ReadDCMImage
CVE-2016-5691
Git commit / fix
These issues were fixed in versions 7.0.1-7 and 6.9.4-5.

ImageMagick heap overflow and out of bounds read

Recently the ImageTragick vulnerability shed some light on the security status of ImageMagick.

This made me wonder how resilient to fuzzing ImageMagick is these days. It's pretty much a posterchild example for a good fuzzing target: Lots of supported complex binary file formats.

I already did some fuzzing on ImageMagick, but as far as I remember that was before I used american fuzzy lop and was done with zzuf. I was also aware that others did some more thorough fuzzing on ImageMagick.

What I did now was relatively simple: I took a trivial, few pixels PNG and used ImageMagick's "convert" tool to convert it into all file formats that have both read and write support in ImageMagick. I used that to run a fuzzing job with afl and asan. By design ImageMagick will sometimes do huge memory allocations, these can be prevented by setting limits for the width, height and memory usage in the policy.xml file.

I discovered one heap buffer overflow in the PICT parser and one heap out of bounds read in the PSD parser. Given how big the attack surface is this is not terrible, but it shows that despite previous efforts there's still potential to fuzz ImageMagick.

Sample file for heap buffer overflow in WritePixelCachePixels() (PICT format)
Git commit / fix

Sample file for heap out of bounds read in PushShortPixel() (PSD format)
Git commit / fix

Both issues have been fixed in the versions 6.9.4-0 and 7.0.1-2. In the meantime new versions (6.9.4-1, 7.0.1-3) came out that, as far as I understand the ChangeLog, remove another potential vector for the ImageTragick vulnerabilities, so you should preferrably update to those.

dosfstools / fsck.vfat: Several invalid memory accesses

I lately fuzzed various filesystem check tools. This uncovered a number of issues in dosfstools / fsck.fat that have now been fixed in the new version 4.0. All issues were found with american fuzzy lop and address sanitizer.

Global out of bounds read file_stat() / check_dir()
Git commit / fix

Unclear invalid memory access in get_fat()
Git commit / fix
CVE-2015-8872

Heap overflow in read_fat()
Heap out of bounds read in get_fat()
Git commit / fix for both issues
CVE-2016-4804

These bugs can pose a security risk if a system automatically checks attached storage media with fsck or in situations where filesystems on untrusted devices get checked. The new version dosfstools 4.0 fixes all four bugs.

PHP EXIF parser out of bounds reads (CVE-2016-4542, CVE-2016-4543, CVE-2016-4544) and a note on custom memory allocators

PHP recently released the security updates 7.0.6, 5.6.21 and 5.5.35 that fix - among a couple of other security issues - a couple of out of bounds issues in the EXIF parser I reported.

Sample file (CVE-2016-4542)
Sample file (CVE-2016-4543)
Sample file (also CVE-2016-4543)
Sample file (CVE-2016-4544)
Bug report
Git commit

These bugs are not exceptionally interesting, but there is something to know when fuzzing PHP and the same applies also to a number of other applications: It uses a custom memory allocator that can sometimes mask issues from memory safety tools like Address Sanitizer. It is therefore good to know about them and disable them during fuzz testing.

With PHP this can be circumvented by setting the environment variable USE_ZEND_ALLOC=0 while fuzzing. This disables the Zend allocator from PHP and uses normal libc memory allocation calls.

I started documenting such issues and workarounds.

Old OpenSSL bug turns out to be a security issue (CVE-2016-2108)

Today OpenSSL released a security advisory. To my surprise one of the high severity issues involved a bug I had reported about a year ago.

The OpenSSL team didn't consider it to be a security issue - and neither did I. What I did back then was to find all possible inputs of the openssl command line tool and test them with american fuzzy lop, including those that very likely wouldn't have any security impact.

The feature I tested while finding this bug is part of the asn1 parser. It's actually not very well documented, but with openssl asn1parse -genconf [input] you can give OpenSSL a text file in a specific format that will then generate an ASN.1 structure. However it seems highly implausible that in any context one would use such a feature with attacker controlled input - therefore the bugs were fixed, but considered to be not security relevant.

But actually, the bug was not in the parser for that format itself. It was in an ASN.1 encoding function. Therefore applications that take ASN.1 encoded data and reencode it may trigger this code.

The bottom line is probably that we should be careful in claims that certain issues are not security relevant. The good news is that OpenSSL fixed this issue anyway, so all versions since June 2015 are safe.

Bugs found with Address Sanitizer in syslog-ng, monit, cmake

Lately a couple of updates have fixed issues that I discovered while testing software with Address Sanitizer (ASAN). I'll publish these combined. Once again these show that a lot of software hasn't been tested with ASAN and what kind of bugs it can uncover.

In syslog-ng a function returns an error -1 that gets stored in an unsigned variable. This cannot be stored and a subsequent check for the error value fails, this later leads to a heap out of bounds memory read. This was fixed in syslog-ng 3.7.3.
Upstream bug report
Pull request / patch
syslog-ng 3.7.3 release notes

In the monit system monitoring tool a heap overflow was discovered. A call to sscanf would write a string of length 256 into a 256 byte long buffer. However this fails to consider the trailing zero byte of that string, therefore causing a one byte heap overflow. This was fixed in monit 5.16.
Commit / fix

In the compilation tool cmake a use after free error was found. This happened becaused a variable was used in a comparison and was free'd during that comparison operation. This was fixed in cmake 3.5.0.
Upstream bug report
Commit / fix

Several out of bounds reads in ProFTPD

The latest releases of ProFTPD 1.3.5a and 1.3.6rc2 fix several out of bounds read issues. I discovered these issues by running the test suite with Address Sanitizer enabled.

An invalid off by one read can happen in the function pr_fs_dircat(). This affects both 1.3.5a and 1.3.6rc1 and earlier.
Upstream bug report
Git commit / fix

An invalid off by one read can happen in the string handling function pr_ascii_ftp_to_crlf(). This code is not present in the stable 1.3.5 release series and only affects 1.3.6 release candidates before rc2.
Upstream bug report
Git commit / fix

A missing null termination of a string causes an out of bounds memory read in a test. This does not affect the ProFTPD code itself, it's just an issue in the test suite.
Upstream bug report
Git commit / fix

Independent of these memory access issues I discovered an issue in ProFTPD regarding Diffie Hellman parameters. The server will choose risky 1024 bit parameters and ignore a user-supplied parameter file.

Heap use after free in Pidgin-OTR plugin (CVE-2015-8833)

The pidgin-otr plugin version 4.0.2 fixes a heap use after free error. The bug is triggered when a user tries to authenticate a buddy and happens in the function create_smp_dialog.

The bug was discovered with Address Sanitizer. This is yet another example why all C/C++ code should be tested with Address Sanitizer enabled.

This bug was already independently discovered and reported by Stefan Sperling in the otr bug tracker.

Independend of this bug another more severe bug in Libotr itself was also disclosed today, please make sure you update both libotr (4.1.1) and the pidgin-otr plugin (4.0.2).

Upstream bug report (contains Address Sanitizer stack trace)
Commit / fix
CVE-2015-8833

Miscomputations of elliptic curve scalar multiplications in Nettle

The Nettle library is a library for basic cryptographic functions. Its most prominent user is GnuTLS.

Through fuzzing of elliptic curve scalar multiplications (multiplying a point on an elliptic curve with a scalar number) I discovered two carry propagation bugs that would lead the cauculations to produce wrong results. They affect the NIST P-256 and P-384 curves. The P-256 bug is in the C code and affects multiple architectures. The P-384 bug is in the assembly code and only affects 64 bit x86. Both bugs were found with the help of american fuzzy lop.

While analyzing these bugs Nettle developer Niels Möller discovered another carry propagation bug in P-256 that was fixed in the same commit (CVE-2015-8805). Nettle 3.2 fixes all three bugs.

The impact is currently unclear, but miscalculations in cryptographic functions should generally be considered security issues. I'd like to encourage cryptographers to try to analyze whether these bugs can lead to cryptographic breaks.

I have published a code example on how to fuzz elliptic curve multiplications. It can compare the output of OpenSSL with either Nettle or NSS. It currently works only with prime field curves, but it can probably be adapted to other curves.

P-256 bug:
Mailing list post with code sample
Commit / fix for P-256 bug
CVE-2015-8803

P-384 bug:
Mailing list post with code sample
Commit / fix for P-384 bug
CVE-2015-8804

Nettle 3.2 release notes