Katie Moussouris on procuring and processing bug reports

In this episode, I talk with Katie Moussouris, founder and CEO of Luta Security. We discuss the five stages of vulnerability disclosure grief, hacking the government, and the pros and cons of bug bounty programs.

Here are some highlights:

The five stages of vulnerability disclosure grief

There are two kinds of reactions we see from organizations that have never received a bug report before. Some of them are really grateful, and that’s ideally where you want people to start, but a lot of them go through what I call the five stages of vulnerability response grief. At first, they are in denial; they say, ‘No, that’s not a bug—maybe you’re mistaken,’ or they get angry and send the lawyers, or they try to bargain with the bug hunter and say, ‘Maybe, if we just did something really stupid and tried to mask what this is, and maybe you won’t talk about it publicly, or tweet about it.’

Then they often get really depressed because they realize this is just one bug report from one bug finder and there might be a ton of bugs they don’t know what to do with. Until finally, they get to the acceptance stage. Ideally, we like it when organizations have gotten to that acceptance stage, when they realize there are bugs in everything, and eventually somebody is going to report a security vulnerability to the organization. Even if you’ve just got a website on the internet, it’s possible that somebody will find and report a security issue to you.

Hacking the government

Hack the Pentagon came about because the U.S. Department of Defense was really interested in hearing about manipulating bug bounty market incentives. Each of those types of bugs would have fetched six figures on the offense market. At the time, Microsoft wasn’t paying six figures per bug for beta bugs—in fact, nobody was—so understanding those market behaviors actually helped the Pentagon feel comfortable in trying out a bug bounty pilot, which is what happened last year.

The results were great for the Pentagon. They got 138 vulnerabilities reported in a 21-day period. They fixed them all within six weeks, I believe. They paid $75,000 in bug bounties to find that many vulnerabilities. Through their usual vendors, it was costing them more than a million dollars a year in federal contracts with different security vendors, and they were typically receiving maybe two or three bug reports a month. There was finally a legal channel for security researchers who wanted to help the government to be able to do so without risking their freedom.

(Editor’s note: Moussouris just helped launch a similar effort with the UK’s National Cyber Security Centre.)

The pros and cons of bug bounties

Anyone can offer cash for bugs. Whether or not it turns out well for them depends on a whole lot of things. Bug bounties can be useful as a focus incentive. If an organization has a pretty good handle on their vulnerabilities and has a process for dealing with the ones they already know about, then that might be a good area to focus on, but I typically don’t think it’s a good way to start. It has been trendy, recently, in the last year or so, as bug bounties have caught on, where company leaders are saying, ‘We’re not getting good vulnerability reports—let’s pay 10 times the bug bounty amounts for a period of time and attract a whole bunch of researchers.’ You might do that, and yes, you might get a whole swarm of bug reports, but are they really the most valuable bugs—the ones that are actually going to help you secure your users, your customers, your enterprise, or your website? Or, are they just going to be a whole swarm of the same bug reported by multiple sources because it was a little bit of a low-hanging-fruit exercise?

I caution people to think through their incentive models. What is it that you really want? Do you want more bug reports? What types of bug reports do you want? How can you structure this so you’re not wasting all your resources and money on an outsourced bug bounty service provider, or on triage provider resources, paying them to sift through reports for you. What would you save by finding these bugs more effectively with a decent security testing program and maybe a full-time person in-house? I talk a lot of people off the bug-bounty ledge, especially if they haven’t done a whole lot of their own homework and testing. Organizations are always going to have competing needs when it comes to spending their security dollars, and I think from a holistic view, bug bounties are not going to be the 100% perfect answer for making people more secure. You cannot “bounty” your way to being secure, the same way you can’t “penetration test” your way to being secure.