How to hack a company by circumventing its WAF for fun and profit – part 3

One of the tools we are more proud of in absolute is Richsploit that we have released few months ago. It allows the exploitation of some EL injection and java deserialization vulnerabilities on multiple versions of RichFaces, an infamous Java library included with many JSF frameworks. Anyway the reason of our pride is not simply connected to the pleasure of being able to execute arbitrary code in a remote system. Also the good time spent to make Richsploit work in certain environments is something that we take into consideration and that is at the basis of the insecurity stories with WAF, and other security devices at layer 7, we are going to share with you today.

To scope or not to scope, that’s the problem

As part of our preparation process for a security test or exercise we normally go through a scoping document together with the customer in order to gain as much information as possible about the environment we will have to test. This is done because sometimes from the list of components, middleware solutions, libraries, etc… so gained, we are able to spot in advance elements useful for the definition of a potential attack path, which saves us a lot of time during the actual security test. The starting point of our story is exactly this. From the scoping document of one of our tests we have identified a component that used the Richfaces 3.x library known to be vulnerable, even though no public exploit existed for it. So, what does the little sympathetic Red Timmy normally do in these cases? Of course he installs that component and with it the exact version of the Richfaces vulnerable library in his local lab, in order to develop and test the exploit over there. Eventually that’s what we have done. Basically, two weeks before the official task’s start date, we had in our hand a full-fledged exploit!

Then it comes the day of the security test in the real customer environment. Same component…same version of Richfaces, but this time the target application is put at the very end of a chain of proxies, load balancers and firewalls of which we knew nothing about. And of course the exploit developed and tested in our lab refuses to work there…

An ASCII character will save us all

Immediately we have started to search the root cause for that, and here what we have discovered. Richfaces implements a custom base64 encoder that does something a normal base64 encoder would never dream to do: it produces in output an encoded object (a string to simplify…) containing one or more exclamation marks. Something like this:

To exploit the vulnerability, the malicious encoded object generated with the Richfaces custom base64 encoder must be passed to the server in the URI of an HTTP GET request, along with the ‘!’ symbol(s). Something like that:

However, for some reason the target application had no intention to serve requests coming with an exclamation mark in the path. Weird! In our local lab everything worked fine. To put it simple, something was preventing our payload to turn out to a beautiful RCE primitive. We had never hated the exclamation mark symbol so much before. A possible explanation for that was that one of the front-end network devices was unhappy with such type of requests. But honestly we will never know, as the case was not investigated at the end of the activity.

Meantime we noticed that the target application used a semicolon character to separate path and querystring, like this:

https://www.bucamitutto.com/appname/path/;jsessionid=DC3DE8B0 [...]

According to RFC 3986a semicolon is often used to delimit parameters and parameter values”.

What if we put a semicolon ‘;’ character in front of the Richfaces encoded object as shown below?

  • The server interprets the encoded object (in purple) not as part of the path but of the query string where the symbol ! is instead allowed. So WAF, load balancers and proxies have nothing to complain.
  • The Richfaces base64 decoding will be still successful, because the semicolon is a non-base64 character, and according to RFC 2045 “Any characters outside of the base64 alphabet are to be ignored”.
  • Because of what the RFC above mandates, the Richfaces base64 decoder will not terminate or throw an error when encounters the semicolon, but skip over it and continue the decoding process as if the character is not there.
  • The decoded malicious payload is then correctly decoded and processed by Richfaces.
  • The payload is executed and RCE condition achieved. Victory!  \o/

The last resort

When one moves from exploiting Richfaces 3.x to Richfaces 4.x, the endpoint to shoot on is instead different compared to the one seen in the previous paragraph. With Richfaces 4.x it is something similar to:

https://www.bucamitutta.com/rfRes/org.richfaces.resource.MediaOutputResource.faces?do=<malicious_encoded_payload>

The request can be sent either as GET or POST: Richfaces handles both methods.

One day the astute Red Timmy faced an unusual (until that moment) situation for him. The application under analysis presented the traces of some resources normally decorating the Richfaces front-end elements, but a F5 Big-IP device blocked any access to the “MediaOutputResource” component.

The device had the Application Security Manager (ASM) and Local Traffic Manager (LTM) modules enabled and a set of custom iRules deployed. The WAF policy was configured in blocking mode, meaning that an error message with a support ID was returned back in case of attack detection.

The custom rules checked that:

  1. the path of each HTTP request did not point to the URL of the “MediaOutputResource” component and the URI did not contain the “do” parameter.
  2. In case of POST request, in addition to the check on the path, the presence of the parameter “do” was verified in the request’s body and not in the URI.

Specifically the rule “1” was applicable to all HTTP methods, not only GET. If the checks were not passed, the request was blocked and an error message returned back to the client.

In addition, differently than how seen in another of our blog posts, no percent-decoding misconfigurations could be appreciated. Everything looked fine. With such a kind configuration in place nothing can go wrong, right?

However, at a certain point we noticed that our target application was exposing the servlet “auth”. It was reachable via POST and took in input the parameter “res” that in the request we intercepted with Burp was pointing to a URL inside the target domain.

For example if “res=https://someserver.somedomain/path?querystring” is provided in the request’s body, as consequence of that the servlet issues a subsequent request like this:

HEAD /path?querystring HTTP/1.1
Host: someserver.somedomain
[...]

Fundamentally, it sends out an HTTP HEAD request based on the content of the parameter “res”. It means we are totally in control of the path, the query string and the target server. We cannot control the HTTP method (all the requests are sent out with the HEAD method by default) but that’s not a problem, as Richfaces 4.x can be exploited through GET which is syntactically similar to HEAD. The only difference is the fact that with HEAD the HTTP body is not returned in the reply, but we will survive to that.

Honestly we are not even “totally” in control of the target server. There is a restriction based on the domain. We could not go outside of the domain we were targeting in that moment (somedomain in the example above). But that was not a problem too, as the Richfaces instance we wanted to exploit was just living into that domain.

Given these premises, what if we set the parameter “res” to the value “https://allowedHost.somedomain/rfRes/org.richfaces.resource.MediaOutputResource.faces?do=<malicious_encoded_payload>” ?

The request crosses the chain of load balancers, firewalls, proxies, etc… completely untouched. Why untouched? Because rule “1” and “2” are not applicable in this case. According to these rules:

  • The path must not point to the URL of MediaOutputResource -> indeed it is set to “/auth”.
  • The URI must not contain the “do” parameter -> in fact there is no “do” parameter.
  • In case of POST request (…and we have a match here) the “do” parameter must not be present in the request’s body. And indeed in the request body there is only the parameter “res”. The value “do” exists only as part of the content pointed to by the “res” parameter (and not as a parameter per se) which is not sufficient condition to block the request.

So, now that we are convinced the request passes through, let’s see what comes next. The request finally lands to the “auth” servlet in the web application server. Here the content of the “res” parameter is extracted and the subsequent HTTP request is issued:

HEAD /rfRes/org.richfaces.resource.MediaOutputResource.faces?do=<malicious_encoded_payload> HTTP/1.1
Host: allowedHost.somedomain
[...]

Due to the WAF configuration mentioned before, this request would be blocked *if* sent from the internet. Indeed the rule “1” would be triggered (the path points to the MediaOutputResource component and the “do” parameter is present in the URI now). However when a malicious request is reflected via SSRF, like in our case, it normally comes from inside the targeted network, ideally localhost if the SSRF endpoint shares the same vhost with the Richfaces component or even a different vhost but configured in the same server. In such cases the request does not cross the WAF(s), load balancers or other security appliances anymore, which are all bypassed (see the picture below).

Indeed after our payload has triggered the vulnerability and we have managed to compromise the target system, a grep for “127.0.0.1” in the web application server’s log files has revealed the following thing:

In simple terms, the HEAD request to “/rfRes/org.richfaces.resource.MediaOutputResource.faces?do=<malicious_encoded_payload>” was clearly coming from localhost, where the vulnerable Richfaces component resided as well, without leaving the targeted system.

And that was another great victory. \o/ \o/

The bottom line

As the Red Timmy’s grandfather always said: “When you need to bypass WAF, the last resort is SSRF”.

These and other WAF policies and signatures bypass techniques (like shell globbing, undefined variables, self-reference folder, etc…) will be taught, along with additional updated content, in our Practical Web Application Hacking Advanced Training on 3-4 August. This year the course will be totally delivered in virtual format. Follow the news in the main page!

And don’t forget to follow us on twitter too: https://twitter.com/redtimmysec

How to hack a company by circumventing its WAF for fun and profit – part 2

So far, one of the most successful and visited posts in our blog has been this one. Following the interest of readers on the topic, we have decided to publish more contents like that. What we will be doing periodically in these spaces is to recount a few stories extrapolated from circumstances that happened for real. We had a lot of fun by living them in first person during penetration testing sessions, red team operations or well deserved bug bounties. We hope you will have too.

The “single-encoded-byte” bypass

Some time ago, we have been contacted to perform the revalidation of a RCE web vulnerability that could be triggered by visiting a URL similar to this:

https://victim.com/appname/MalPath/something?badparam=%rce_payload%

The application was hosted behind a Big-IP device with ASM (Application Security Manager) module enabled, a layer 7 web application firewall (WAF) available on F5’s BIG-IP platforms. To react immediately and not leave the solution exposed to the internet, the network engineers thought well of deploying a custom WAF rule to protect the endpoint before a formal fix was actually introduced in the application’s code. Our role consisted of investigating the effectiveness and robustness of this rule which looked like something similar to this:

The meaning of it was that in case a HTTP GET request had been intercepted (line 2) and the lowercase-converted URI contained both the strings “/malpath/” and “badparam=” (lines 3 and 4), the request would have been logged first (line 7) and then blocked by returning the HTTP error 404 to the client (line 8).

Actually, after deploying the custom WAF rule, by visiting the URL:

https://victim.com/appname/MalPath/something?badparam=%rce_payload%

…we receive the 404 HTTP error, meaning that the rule is working and the WAF is doing properly its job.

However, by reading the F5 documentation we discovered that the directive “HTTP::uri” (lines 3 and 4) returns the URI of the request without first fully decoding it! So if we visit the URL:

https://victim.com/appname/%4dalPath/something?badparam=%rce_payload%

…with “%4d” being the URL-encoded form of the character ‘M’, this is what happens:

  • The WAF intercepts the GET request. The URI in there “/appname/%4dalPath/something?badparam=%rce_payload%” is first entirely converted to lowercase because of the “tolower” function in the WAF rule, and becomes “/appname/%4dalpath/something?badparam=%rce_payload%”.
  • Then the URI is looked for the presence of the strings “/malpath/” and “badparam=”. Anyway, in the first case the string “/%4dalpath/” we sent does not match with the string “/malpath/” in the WAF rule (line 3), which hence is not hit.
  • Once the WAF control is bypassed, the HTTP request reaches the backend server where the URI canonicalization and decoding processes take place. So the request containing the URI converted from “/appname/%4dalPath/something?badparam=%rce_payload%” to “/appname/MalPath/something?badparam=%rce_payload%” is passed to the target web  application which in turn serves it.
  • The “%rce_payload%” is now executed and produces its effects (opening of a reverse shell, invocation of an operating system command, etc…)

Here the lesson learnt of the day is: always ensure that URI is decoded before comparing and checking it. Absolute lack of URI decoding is very bad. The usage of “contains” keyword in a WAF rule is even worse.

Partial URL decoding = full bypass

A couple of days later we were called back again to carry out the revalidation of the revalidation (lol) as the fix in the application code was taking more time than expected.

This time visiting both the URLs:

https://victim.com/appname/MalPath/something?badparam=%rce_payload%

and

https://victim.com/appname/%4dalPath/something?badparam=%rce_payload%

returned the 404 error page. In simple words, the WAF blocked us.

Then we attempted to double encode the letter “M” of “/Malpath/” with the sequence “%254d”. The final URL to visit was:

https://victim.com/appname/%254dalPath/something?badparam=%rce_payload%

Nothing. The web application firewall was blocking this request too. The reimplementation of the WAF rule now looks something like this:

Here the lines from 15 to 24 are equivalent to the lines 1-10 of the previous rule’s version. However this time the strings “/malpath/” and “badparam=” are checked against a variable called “$decodeUri” (lines 17 and 18) and not “HTTP::uri” as done before. What is this and where does it come from? Actually the lines from 1 to 14 are defining its content. In simple terms the “$decodeUri” variable is just the URI of the request (“[HTTP::uri]” at line 2). It is URI-decoded at most “max” times (with “max” being defined at line 4) through a “while” loop (lines 9-13) by using a counter variable (“cnt” declared at line 5). The loop is interrupted in case there are no more encoded characters left or the iteration in the cycle has occurred for “max” times (with “cnt” increased during each iteration).

To better understand what’s going on here, it could be helpful to browse this page and perform fourfold percent-encoding of the letter “M” by writing the sequence “%252525254d” into the text box. Now press over the “Decode” button (see picture below) for four times and during each pause see what happens. Have you seen it? This is exactly what the WAF rule above is doing with our URI.

So to bypass it, we have simply visited the following URL:

https://victim.com/appname/%252525254dalPath/something?badparam=%rce_payload%

On reception the WAF decodes the URI as follows:

  • (1st iteration of the loop):
    /appname/%2525254dalPath/something?badparam=%rce_payload%
  • (2nd iteration of the loop):
    /appname/%25254dalPath/something?badparam=%rce_payload%
  • (3rd iteration of the loop):
    /appname/%254dalPath/something?badparam=%rce_payload%
  • (4th iteration of the loop):
    /appname/%4dalPath/something?badparam=%rce_payload%

When the WAF rule quits the loop, the request’s URI is set to “/appname/%4dalPath/something?badparam=%rce_payload%“. Again, as in the previous case, the string “/malpath/” (line 17 of the WAF rule) does not match with the one in the URI “/%4dalpath/”, so the WAF let the request pass through. The control is bypassed once more and the effect is always the execution of “%rce_payload%”. GAME OVER for the second consecutive time.

One could think: “Ok, but this is a stupid mistake! Just fire these monkeys preteding to be security guys!”. This is not totally true as the “-normalized” option, that undoubtedly simplifies the life of network engineers, can be adopted only with the most recent versions of iRules, leaving as a unique option the creation and deployment of a custom rule which is notoriously an error-prone process. However, it is still surprising how many sources out there are publicly advertising these flawed examples, even in the official F5 documentation (watch the screenshot below).

For copy & paste supporters this kind of “code snippets” are like honey. Don’t get us wrong. We have nothing against copy & paste, except when this practice is performed in cybersecurity without understanding what is being done. In such circumstances copy&paste has been a countless number of times the root cause of many serious security vulnerabilities.

But let’s switch back to our main story! After the identification of the new issue the fix was quick (see the added part below).

Basically, following the “while” cycle (lines 9-13 in the picture at the beginning of this paragraph), the content of the WAF rule was updated with an “if” branch. It checks, at the end of the loop, if the value of the counter variable “cnt” is equal to the value of the “max” variable. In case it is, this means that the “while” cycle was interrupted while there were still characters to decode left in the URI. As we don’t want to process a request still containing encoded characters, the WAF rule responds with a 404 error, preventing the activation of the malicious payload.

Wrong assumptions … any gumption?

Most misconfigurations and bypass techniques at WAF level are not linked to technical problems or bugs afflicting the device itself, but wrong assumptions in the logic of its rules.

Our story may have had a happy ending at this point. Two revalidations had already revealed some flaws in the web application firewall’s configuration and finally a pretty tough WAF rule had been implemented, decently decoding the URI. To make it short, in a certain moment somebody realizes that the same vulnerability could have been exploited with a HTTP POST request.  The WAF rule provided protection only against GET requests and a new modification to it was needed. So, after the umpteenth change we were called back again to verify that this time everything was working as expected.

To cover both exploitation cases (GET and POST) the network engineers thought to apply a fix by deleting the line 16 of the WAF rule:

if ([HTTP::method] equals “GET”) [...]

Without that line the check on the URI would have been performed on any HTTP request, regardless of the method specified. It could have been an elegant solution if had worked…but can you spot the problem here? Take yourself few seconds before to answer. Well, when the URL below is browsed:

https://victim.com/appname/MalPath/something?badparam=%rce_payload%”

the following HTTP request is issued:

GET /appname/MalPath/something?badparam=%rce_payload% HTTP/1.1
[...]

If after discovering that the GET method is blocked at WAF level an attacker attempted to circumvent it via POST, the following request would be instead issued:

POST /appname/MalPath/something HTTP/1.1
[other HTTPheaders]

badparam=%rce_payload%

Would the “%rce_payload%” be triggered this time, given all the premises above? Yes, it would. Why? The answer is in the lines 17 and 18 of the WAF rule, shown below once more.

Let’s compare the content of the attacker’s POST request with the logic of the custom rule. In this case, at line 17 the fully decoded and lowercase-converted URI contains the string “/malpath/”. True. However the line 18 stems from an incorrect assumption. The parameters of a POST request reside in the HTTP body and not in the query string! The line 18 of the WAF rule is instead checking for the presence of the string “badparam=” in the “$decodedUri” variable which in the case of the attacker’s POST request returns the string “/appname/MalPath/something”. For sake of clarity, the WAF rule is not checking the string “badparam=” where it should, so the rule is bypassed.

Officially the revalidation of the revalidation of the revalidation was last round of check to be confident enough that the WAF rule implemented was providing the level of security desired against this web vulnerability. It took 1 week to tune up the custom WAF rule and then, suddenly, few more days later, it was removed because in the meantime the application code had been patched 😀

If it helps to repeat that once more, WAF virtual patching is ok if you need a bit of extra time before fixing a serious vulnerability, but it must not be abused. One week of exposure to RCE is a huge period of time for motivated attackers. The only way to sleep peacefully is to solve your problems at code level and patch it, patch it, patch it as soon as possible! Probably would have been worth from the beginning to dedicate all the resources to patch the application code than struggling to implement a working WAF rule by trial and error.

That’s all for today. Other stories like this will be published in future. So stay tuned and continue to follow us!

Hey! Red Timmy Security will be at Blackhat Las Vegas this year too! Check out our trainings!

Practical Web Application Hacking Advanced 1-2 August and 3-4 August 2020.

How to hack a company by circumventing its WAF through the abuse of a different security appliance and win bug bounties

Hey, wait! What do bug bounties and network security appliances have in common? Usually nothing! On the contrary, the security appliances allow virtual patching practices and actively participate to reduce the number of bug bounties paid to researchers…but this is a reverse story: a bug bounty was paid to us thanks to a misconfigured security appliance. We are not going to reveal neither the name of the affected company (except it was a Fortune 500) nor the one of the vulnerable component. However, we will be talking about the technique used, because it is astonishingly of disarming simplicity.

The observation

All has begun by browsing what at that time we did not even know yet to be a valuable target, let us call it “https://targetdomain“. Almost by accident, we noticed that a subdomain responsible for the authentication on that website had exposed some CSS and Javascript resources attributable to a Java component well known to be vulnerable to RCE (Remote Code Execution).

The weird thing was that by browsing the affected endpoint (something like “https://auth.targetdomain/vulnerable_endpoint?param=malicious_RCE_payload“) we received a HTTP 404 reply from the server, which made us suspect the presence of a Web Application Firewall. Why that particular endpoint should not be reachable if the resources decorating it (like .css and .js files) are? This clearly made us believe we were in front of a WAF. After a few more requests, all blocked, we confirmed some kind of WAF rule was indeed preventing us from reaching the target endpoint.

The “weird” thing

By browsing one of the applications hosted (i.e. https://targetdomain/appname) we are invited to authenticate to “https//auth.targetdomain“. During the authentication process, we notice another strange thing. At a certain moment we are redirected to a URL like:

https//targetdomain/?cfru=aHR0cHM6Ly90YXJnZXRkb21haW4vYXBwbmFtZQ==

with “aHR0cHM6Ly90YXJnZXRkb21haW4vYXBwbmFtZQ==” being clearly a base64-encoded string. The base64 payload, after decoding, showed to be nothing more that the URL we had originally requested access to before starting the authentication, that is “https://targetdomain/appname“.

But what actually that “cfru” parameter was? Some quick research online shows it is part of the Bluecoat web filtering technology, a notorious proxy server appliance. So, this told us a bit more about the remote infrastructure. The HTTP requests we send to the target cross at least one WAF device and a Bluecoat proxy server before reaching the front end web servers and application servers, like reconstructed below.

The idea

A light bulb has lit up on our head once we discovered that this “cfru” parameter was publicly accessible, namely no authentication to the portal was required to pass our payload to it. Therefore we started to base64-encode URLs of external domains under our control and feed the “cfru” parameter with these strings.  The hope was to trigger some kind of SSRF. What we got at the end was much better.

Unfortunately, at that specific moment in time, we did not receive back any HTTP requests. However, in our internet-exposed machines, we could see the DNS resolution process started from “targetdomain”. It seemed the TCP outgoing connections from target website were prohibited.  The only authorized thing was, as said, DNS traffic. Then instead of trying to trigger SSRF requests to external hosts we turned our attention to internal subdomains (https://auth.targetdomain, https://blog.targetdomain, https://www.targetdomain, etc…).

We start to base64-encode few of these URLs into the “cfru” parameter and almost immediately notice another weirdness. For some URLs we get a HTTP 302 redirect back. For some others we do not. In this latter case instead the HTTP body in the reply contains the HTML code of the requested resource, as if Bluecoat forwarded the request to the destination resource and returned its content back to us by acting as a web proxy. Most importantly, this behavior was observed also when we encoded in the “cfru” parameter the subdomain responsible for the authentication to the portal (https//auth.targetdomain), namely the one we believed was hosting a Java component vulnerable to RCE.

The turning point

Here was the turning point! We have made the following assumption. If the resource

https://auth.targetdomain/vulnerable_endpoint?param=malicious_RCE_payload

is directly browsed, our HTTP request lands immediately to the WAF, where there is configured a rule that recognizes the malicious attempt (the malicious payload pointed to by “param“) and sends back a HTTP 404 error, in fact blocking the attack.

But what if we encode in base64 the URL

https://auth.targetdomain/vulnerable_endpoint?param=malicious_RCE_payload

which produces the following base64 string

aHR0cHM6Ly9hdXRoLnRhcmdldGRvbWFpbi92dWxuZXJhYmxlX2VuZHBvaW50P3BhcmFtPW1hbGljaW91c19SQ0VfcGF5bG9hZA==

and pass it to the “cfru” parameter as follows?

https//targetdomain/?cfru=aHR0cHM6Ly9hdXRoLnRhcmdldGRvbWFpbi92dWxuZXJhYmxlX2VuZHBvaW50P3BhcmFtPW1hbGljaW91c19SQ0VfcGF5bG9hZA==

In our case:

  1. The request crossed the WAF which had nothing to complain.
  2. Then it arrived to the Bluecoat device that in turn base64-decoded the “cfru” parameter and issued a GET request toward the internal host https://auth.targetdomain/vulnerable_endpoint?param=malicious_RCE_payload.
  3. This in turn triggered the vulnerability.

Bingo!

And bingo! We can see the output of our malicious payload (nothing more than the “hostname” command) exfiltrated via DNS (outgoing TCP connections to our host located in the internet were indeed blocked as already said previously).

Furthermore, we played a bit with our malicious payload in order to have the output of our injected commands returned directly as part of the HTTP headers in the server reply.

Conclusion

There are at least two mistakes that can be spot here:

  • The bluecoat device was behaving as a request “forwarder” instead of responding with a HTTP redirect as happened for other URLs (that would have caused the subsequent client requests to be caught and blocked by WAF).
  • There was no rule implemented at WAF level that base64-decoded the “cfru” parameter before passing it to the Bluecoat device, in order to analyze whether or not the request’s content matched with one of the blocking rules deployed in the WAF itself.

Good for us! We notified the vulnerability to the vendor straight away and they decided to recognize us a nice bug bounty!

The bottom line here is that virtual patching is ok if you need a bit of extra time before fixing a serious vulnerability. But if you use it in place of real patching, well it is only question of time before you will get hacked.

If you want to know more about similar exploitation techniques and other web hacking tricks, check out our Blackhat Las Vegas courses on 1-2 August and 3-4 August 2020, because this will be one of the topics covered there.