Thursday, March 22, 2012

On the failings of Pwn2Own 2012

This year's Pwn2Own and Pwnium contests were interesting for many reasons. If you look at the results closely, there are many interesting observations and conclusions to be made.

$60k is more than enough to encourage disclosure of full exploits

As evidenced by the Pwnium results, $60k is certainly enough to motivate researchers into disclosing full exploits, including sandbox escapes or bypasses.

There was some minor controversy on this point leading up to the competitions, culminating in this post from ZDI. The post unfortunately was a little strong in its statements including "In fact, we don't believe that even the entirety of the $105,000 we are offering would be considered an acceptable bounty", "for the $60,000 they are offering, it is incredibly unlikely that anyone will participate" and "such an exploit against Chrome will never see the light of day at CanSecWest". At least we all now have data; I don't expect ZDI to make this mistake again. Without data, it's an understandable mistake to have made.

Bad actors will find loopholes and punk you

One of the stated -- and laudable -- goals of both Pwn2Own and Pwnium is to make users safer by getting bugs fixed. As recently noted by the EFF, there are some who are not interested in getting bugs fixed. At face value, it would seem to be counterproductive for these greyhat or blackhat parties to participate.

Enter VUPEN, who somehow managed to turn up and get the best of all worlds: $60k, tons of free publicity for their dubious business model and... minimal cost. To explore the minimal cost, let's look at one of the bugs they used: a Flash bug (not Chrome as widely reported), present in Flash 11.1 but already fixed in Flash 11.2. In other words, the bug they used already had a fixed lifetime. Using such a bug enabled them to collect a large prize whilst only handing over a doomed asset in return.

Although operating within the rules, their entry did not do much to advance user security and safety -- the bug fix was already in the pipeline to users. They did however punk $60k out of Pwn2Own and turned the whole contest into a VUPEN marketing spree.

Game theory

At the last minute at Pwn2Own, contestants Vincenzo and Willem swooped in with a Firefox exploit to collect a $30k second place prize. The timing suggests that they were waiting to see if their single 0-day would net them a prize or not. It did. We'll never know what they would have done if the $30k reward was already sewn up by someone else, but one possibility is a non-disclosure -- which wouldn't help make anyone safer.

Fixing future contests

The data collected suggests some possible structure to future contests to ensure they bring maximal benefit to user safety:
  • Require full exploits, including sandbox escapes or bypasses.

  • Do not pay out for bugs already fixed in development releases or repositories.

  • Have a fixed reward value per exploit.

9 comments:

saso said...

and the firefox bug was actually also already known and fixed...

https://www.mozilla.org/security/announce/2012/mfsa2012-19.html

Anonymous said...

You forget to mention one important thing.

The only people participating in PWNIUM were: someone from a poor poor country where 60k USD is a lot of money and the other one, whose identity you keep hidden, is a student.

While both might be security researchers they do not have well paid jobs in the security industry. Fact is that everybody with a real job in IT security did not go for the 60k USD.

Chris Evans said...

@Anonymous: thanks for the comment! I actually had some interesting exchanges with researchers (with real jobs ;-) after Pwnium. One guy thought he might be able to do something at the $40k level and the other one the $60k level. The logistics simply weren't in line for them this time.

Anonymous said...

@Anonymous: Vincenzo and Willem both have good paying jobs in the security industry and they decided to go for the 30k USD in Pwn2Own. And they're both smarter than 95% of the cats out there; so some people with a real job in IT security do go for less than 60k USD.

Anonymous said...

There is no upper limit could be set here. Some people would go for 60k but some others would not. FYI, I sold a reliablel 0day Firefox exploit for 120k last month.

Jimmy Bergman said...

Another concern is that Google sponsoring $60k for sandbox escapes with Pwnium for sure means that noone sane would ever submit them at $3133.70 in the rewards program. Regardless if $60k is enough, $3133.70 isn't - especially now that you set the bar higher.

Instead people that do think $60k is worth it will wait for the next year, causing people to be unprotected for a lot longer than if the $60k-for-sandbox-escape reward was permanent as the rewards program.

dragosr said...

I have this theory, that, no matter what the prize is, some exploit researcher (with a 37.8% chance of it being the grugq) will complain that the market value of the exploit is much higher than the prize offered in an attempt to inflate the estimation of exploit pricing - regardless of whether that researcher actually has an example of said exploit or not.

Anonymous said...

perhaps i'm misreading these infosec business marketting scams (see: pissing match) which seem to be about whose software is better in protecting the world from unknown adversaries. however, i thought the concept of these contests was to show that you can be hacked regardless of platform. due to how cheap exploitation has gotten over the years, seeing the skills of different defense contractors compete in an exploit-writing contest is a win as now there's more information out there. there's now content for media people such as google to be able to discuss on. especially now since this "trade" has escalated into a different order of business within the past couple of years.

having more of this type of data actually opens up a chance to know your enemies (like csoghian is able to write about). and, it's incredibly useful data if you know that your enemy commonly does "research" at a national level. the end-goal appears to be a live demonstration of an arms race in a format that people could be entertained in. but likewise, it would be just as entertaining if representatives from non-democratic locations would be interested in demonstrating their skillsets/pockets.

Anonymous said...

I'd hope the point of the contest was more than "to show that you can be hacked regardless of platform". As any 2-bit technologist will tell you, this is true even without entertaining examples. We don't need a contest for that, actually I'd hope no such contest existed. As it's essentially handing out cash prizes, for the purpose of preparing live exploits. Which would remain unpatched, until caught later in their cycle. Some might not have been developed at that time, giving the loopholes more time to be closed. Full-Disclosure is the only way the contest is truly beneficial. AT least for reasons other than entertainment, excluding the benefit of dark organizations.