Skip to content

Optionsbleed - HTTP OPTIONS method can leak Apache's server memory

optionsbleedIf you're using the HTTP protocol in everday Internet use you are usually only using two of its methods: GET and POST. However HTTP has a number of other methods, so I wondered what you can do with them and if there are any vulnerabilities.

One HTTP method is called OPTIONS. It simply allows asking a server which other HTTP methods it supports. The server answers with the "Allow" header and gives us a comma separated list of supported methods.

A scan of the Alexa Top 1 Million revealed something strange: Plenty of servers sent out an "Allow" header with what looked like corrupted data. Some examples:
Allow: ,GET,,,POST,OPTIONS,HEAD,,
Allow: POST,OPTIONS,,HEAD,:09:44 GMT
Allow: GET,HEAD,OPTIONS,,HEAD,,HEAD,,HEAD,, HEAD,,HEAD,,HEAD,,HEAD,POST,,HEAD,, HEAD,!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"
Allow: GET,HEAD,OPTIONS,=write HTTP/1.0,HEAD,,HEAD,POST,,HEAD,TRACE

That clearly looked interesting - and dangerous. It suspiciously looked like a "bleed"-style bug, which has become a name for bugs where arbitrary pieces of memory are leaked to a potential attacker. However these were random servers on the Internet, so at first I didn't know what software was causing this.

Sometimes HTTP servers send a "Server" header telling the software. However one needs to be aware that the "Server" header can lie. It's quite common to have one HTTP server proxying another. I got all kinds of different "Server" headers back, but I very much suspected that these were all from the same bug.

I tried to contact the affected server operators, but only one of them answered, and he was extremely reluctant to tell me anything about his setup, so that wasn't very helpful either.

However I got one clue: Some of the corrupted headers contained strings that were clearly configuration options from Apache. It seemed quite unlikely that those would show up in the memory of other server software. But I was unable to reproduce anything alike on my own Apache servers. I also tried reading the code that put together the Allow header to see if I can find any clues, but with no success. So without knowing any details I contacted the Apache security team.

Fortunately Apache developer Jacob Champion digged into it and figured out what was going on: Apache supports a configuration directive Limit that allows restricting access to certain HTTP methods to a specific user. And if one sets the Limit directive in an .htaccess file for an HTTP method that's not globally registered in the server then the corruption happens. After that I was able to reproduce it myself. Setting a Limit directive for any invalid HTTP method in an .htaccess file caused a use after free error in the construction of the Allow header which was also detectable with Address Sanitizer. (However ASAN doesn't work reliably due to the memory allocation abstraction done by APR.)

FAQ

What's Optionsbleed?

Optionsbleed is a use after free error in Apache HTTP that causes a corrupted Allow header to be constructed in response to HTTP OPTIONS requests. This can leak pieces of arbitrary memory from the server process that may contain secrets. The memory pieces change after multiple requests, so for a vulnerable host an arbitrary number of memory chunks can be leaked.

The bug appears if a webmaster tries to use the "Limit" directive with an invalid HTTP method.

Example .htaccess:

<Limit abcxyz>
</Limit>

How prevalent is it?

Scanning the Alexa Top 1 Million revealed 466 hosts with corrupted Allow headers. In theory it's possible that other server software has similar bugs. On the other hand this bug is nondeterministic, so not all vulnerable hosts may have been caught.

So it only happens if you set a quite unusual configuration option?

There's an additional risk in shared hosting environments. The corruption is not limited to a single virtual host. One customer of a shared hosting provider could deliberately create an .htaccess file causing this corruption hoping to be able to extract secret data from other hosts on the same system.

I can't reproduce it!

Due to its nature the bug doesn't appear deterministically. It only seems to appear on busy servers. Sometimes it only appears after multiple requests.

Does it have a CVE?

CVE-2017-9798.

I'm seeing Allow headers containing HEAD multiple times!

This is actually a different Apache bug (#61207) that I found during this investigation. It causes HEAD to appear three times instead of once. However it's harmless and not a security bug.

Launchpad also has a harmless bug that produces a malformed Allow header, using a space-separated list instead of a comma-separated one.

How can I test it?

A simple way is to use Curl in a loop and send OPTIONS requests:

for i in {1..100}; do curl -sI -X OPTIONS https://www.google.com/|grep -i "allow:"; done

Depending on the server configuration it may not answer to OPTIONS requests on some URLs. Try different paths, HTTP versus HTTPS hosts, non-www versus www etc. may lead to different results.

Please note that this bug does not show up with the "*" OPTIONS target, you need a specific path.

Here's a python proof of concept script.

What shall I do?

If you run an Apache web server you should update. Most distributions should have updated packages by now or very soon. A patch can be found here. A patch for Apache 2.2 is available here (thanks to Thomas Deutschmann for backporting it).

Unfortunately the communication with the Apache security team wasn't ideal. They were unable to provide a timeline for a coordinated release with a fix, so I decided to define a disclosure date on my own without an upstream fix.

If you run an Apache web server in a shared hosting environment that allows users to create .htaccess files you should drop everything you are doing right now, update immediately and make sure you restart the server afterwards.

Is this as bad as Heartbleed?

No. Although similar in nature, this bug leaks only small chunks of memory and more importantly only affects a small number of hosts by default.

It's still a pretty bad bug, particularly for shared hosting environments.

Updates:

Analysis by Apache developer William A. Rowe Jr.

Distribution updates:
Gentoo: Commit (2.2.34 / 2.4.27-r1 fixed), Bug
NetBSD/pkgsrc: Commit
Guix: Commit
Arch Linux: Commit (2.4.27-2 fixed)
Slackware: Advisory
NixOS: Commit
Debian: Security Tracker, Advisory (2.4.10-10+deb8u11, 2.4.25-3+deb9u3)
Ubuntu: Advisory (2.4.25-3ubuntu2.3, 2.4.18-2ubuntu3.5, 2.4.7-1ubuntu4.18)

Media:
Apache-Webserver blutet (Golem.de)
Apache Webserver: "Optionsbleed"-Bug legt Speicherinhalte offen (heise online)
Risks Limited With Latest Apache Bug, Optionsbleed (Threatpost)
Apache “Optionsbleed” vulnerability – what you need to know (Naked Security)
Apache bug leaks contents of server memory for all to see—Patch now (Ars Technica)

Trackbacks

The Fuzzing Project on : How Optionsbleed wasn't found in 2014

Show preview
Shortly after I published details about the Optionsbleed bug I learned about something quite surprising: Others had already discovered this bug before, but have neither pinned it down to Apache nor recognized that it is a security vulnerability. A pape

Hanno's blog on : Introducing Snallygaster - a tool to scan for secrets on web servers

Show preview
A few days ago I figured out that several blogs operated by T-Mobile Austria had a Git repository exposed which included their wordpress configuration file. Due to the fact that a phpMyAdmin installation was also accessible this would have allowed me to c

Comments

Display comments as Linear | Threaded

lbiegaj on :

I was able to reproduce this issue on Apache 2.2.22 with a legitimate method:

(angle bracket open)Limit GET(angle bracket close)
Allow from x.y.z.v
Deny from all
(angle bracket open)/Limit(angle bracket close)

for I in `seq 1 100`; do curl -sI -X OPTIONS http://myhost/ | grep ^Allow; done | uniq
Allow: ,GET,HEAD,POST,OPTIONS
Allow: GET,HEAD,POST,OPTIONS
Allow: ,GET,HEAD,POST,OPTIONS
Allow: GET,HEAD,POST,OPTIONS

lbiegaj on :

Seems that this blog filtered out "LIMIT GET" tags from my comment.

easteregg on :

same with 2.4.25-3+deb9u2

parseword on :

For those who can reproduce, if you don't mind testing, I'm curious whether blocking the OPTIONS verb via mod_allowmethods works as a temporary mitigation.

In httpd.conf:
LoadModule allowmethods_module modules/mod_allowmethods.so

Inside your Directory stanzas:
#Disable unwanted methods (OPTIONS, PUT, PROPFIND, etc.)
AllowMethods GET POST HEAD

This should cause any OPTIONS request to return a status of "HTTP/1.1 405 Method Not Allowed" and emit an empty "Allow:" header. I can't reproduce the bug on any of my instances to ensure this properly scrubs the "Allow:" header, though.

Noel Whitemore on :

That particular module is still termed "experimental" and there's no mention in the official documentation of what the expected output should be, so you'd probably have to look at the source code to see what's going on behind the scenes. On a separate issue, deliberately limiting the OPTIONS method in the normal way (as Google appears to be doing) doesn't work because when you send a request to Google's servers you get the "HTTP/1.1 405 Method Not Allowed" message but the accepted methods are still listed on the next line of the header.

William A Rowe Jr on :

Noel is correct with respect to blocking OPTIONS.

The other reason that blocking OPTIONS is no solution is that the corruption itself in the shared startup configuration list of supported methods still occurs on any request which parses the suspect .htaccess file (where an unrecognized method is given by accident or deliberately.) Corrupting that shared configuration data will still lead to unpredictable behavior.

Lee on :

Good article. Thanks! You may want to fix a few mistakes I found while reading it.

However, ASAN doesn't work "reliable"... > reliably
...so not all vulnerable hosts may have been "catched"... > caught
... you should drop everything "you do now"... > you are doing now

Sawood Alam on :

I have noted this issue in 2014 while I was working on my paper, "Support for Various HTTP Methods on the Web" (https://arxiv.org/pdf/1405.2330.pdf). My observations about this issue as well as some other problems are described in the section 5.3 of the paper. However, I didn't look into it from the security perspective at that time.

Jordan on :

So, if I don't use the Limit directive in an htaccess file, then I'm safe from this? Is it really that commonly used? What about if it's in the global configuration?

JoeRandomHacker on :

RewriteEngine On
RewriteCond %{REQUEST_METHOD} !(GET|POST)
RewriteRule .* - [F]

Manju on :

Does this vulnerability affects Apache Tomcat 8.5 versions?

JoeRandomHacker on :

No, this is only about apache httpd, not about other projects of the apache foundation like tomcat.

Colin Watson on :

I've just deployed the fix for the minor launchpad.net bug identified as part of this.

(I wonder if I'm the only one who thinks "hmm, bug report from Hanno; I wonder what interesting widespread problem they're researching"?)

William A Rowe Jr on :

Note that the new Apache directive RegisterHttpMethod lets you pre-define unusual method strings to avoid this defect.

in the httpd.conf global/startup config;
RegisterHttpMethod ONE TWO THREE

in the .htaccess files;


or similar. If you grant untrusted users .htaccess editing permission, you are always opening up the possibility of malicious configuration.

William A Rowe Jr on :

(The eaten .htaccess example I offered was...)
-LT- Limit POST DELETE TWO -GT-

Hayden James on :

cPanel has released update for this via easyapache. Esp for shared hosting using cPanel/WHM you should update asap.

--------------

SUMMARY
cPanel, Inc. has released updated RPMs for EasyApache 4 and EasyApache 3.34.17 on September 20, 2017, with a patched versions of Apache 2.2 and 2.4 to address the optionsbleed vulnerability related to CVE-2017-9798. We strongly encourage all Apache 2.2 and 2.4 users to upgrade their system and obtain the patch.


AFFECTED VERSIONS
All versions of Apache 2.4 through 2.4.27
All versions of Apache 2.2 through 2.2.34


SECURITY RATING
The National Vulnerability Database (NIST) has given the following severity ratings to these CVEs:


CVE-2017-9798 - HIGH

Apache 2.4.27-8
Patched optionsbleed vulnerability related to CVE-2017-9798

Apache 2.2.34
Patched optionsbleed vulnerability related to CVE-2017-9798

SOLUTION
cPanel, Inc. has released EasyApache 3.34.17 with updated versions of Apache 2.2 and Apache 2.4. Unless you have disabled EasyApache updates, the EasyApache application updates to the latest version when launched. Run EasyApache to rebuild your profile with the latest version of Apache.

cPanel, Inc. has released updated RPMs for EasyApache 4 on September 20, 2017, with an updated versions of Apache 2.4. Unless you have enabled automatic RPM updates in your cron, update your system with either yum update or WHM's Run System Update interface.

Add Comment

Enclosing asterisks marks text as bold (*word*), underscore are made via _word_.
Standard emoticons like :-) and ;-) are converted to images.
E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.
CAPTCHA

Form options