Mozilla NSS: Wrong calculation results in mp_div() and mp_exptmod()
A bug in the NSS library can cause certain cryptographic calculations to produce wrong results. The bug is in the function mp_div(), this function gets used by the function mp_exptmod(), a combination of an exponentiation and a modulo operation. The mp_exptmod() function is used in several cryptographic algorithms, most notably it's the function to sign and encrypt with RSA and to calculate Diffie Hellman key exchanges. The NSS library is used for TLS connections in various Mozilla products.
This bug was found by comparing the results of NSS and OpenSSL bignum calculations and fuzzing the input with american fuzzy lop. A similar bug was also found and fixed in OpenSSL recently.
I first reported both issues separately, not knowing that they were caused by the same bug. In a first analysis both the Mozilla developers and I thought that the impact of the mp_div() bug would be minor, because it wasn't used in the cryptography code of NSS. But we overlooked that it was indirectly used through the mp_exptmod() function.
It is unclear what the exact impact and severity of this bug is. I am not aware of a practical way to exploit it, but as it affects important crypto code it might be possible to find exploitable scenarios.
I have published code examples with test inputs triggering the bug for both functions on Github. This bug is fixed in NSS 3.21 and in Firefox 44.
Mozilla Foundation Security Advisory 2016-07
Commit with the fix
Bug report for mp_div() (still private)
Bug report for mp_exptmod() (still private)
Example for mp_div()
Example for mp_exptmod()
CVE-2016-1938
This bug was found by comparing the results of NSS and OpenSSL bignum calculations and fuzzing the input with american fuzzy lop. A similar bug was also found and fixed in OpenSSL recently.
I first reported both issues separately, not knowing that they were caused by the same bug. In a first analysis both the Mozilla developers and I thought that the impact of the mp_div() bug would be minor, because it wasn't used in the cryptography code of NSS. But we overlooked that it was indirectly used through the mp_exptmod() function.
It is unclear what the exact impact and severity of this bug is. I am not aware of a practical way to exploit it, but as it affects important crypto code it might be possible to find exploitable scenarios.
I have published code examples with test inputs triggering the bug for both functions on Github. This bug is fixed in NSS 3.21 and in Firefox 44.
Mozilla Foundation Security Advisory 2016-07
Commit with the fix
Bug report for mp_div() (still private)
Bug report for mp_exptmod() (still private)
Example for mp_div()
Example for mp_exptmod()
CVE-2016-1938