My Security Planet » Items by Brian Chess

Items by Brian Chess

Fortify blog

Penetration Testing is Dead, Long Live Penetration Testing

Posted: December 15th, 2008, 11:36pm CST by Brian Chess

Tags: [edit]

Last week, in a CSO magazine article titled Penetration Testing: Dead in 2009, I said that penetration testing is undergoing a transformation—it’s going to die and be reborn as part of a more tightly integrated approach to security. This week Ivan Arce responded in defense of penetration testing in a CSO article titled Twelve Reasons Pen Testing Won't Die. Ivan is a class act, which is why his article isn’t titled "12 Reasons Brian Chess is a Ninny." But Ivan doesn’t address the core of my original argument, so I’m going to further explain myself here. Let me start by saying this: if you misread the original article to mean "on Jan 1, 2009 all penetration testers are going to go out Jonestown-style", then you need to keep reading. It’s December. For me that usually means my travel schedule winds down, I spend more time doing family stuff, and I start thinking about what I’m going to do next year. This year the travel hasn’t let up, and as for the future, all I can think about are the things Gary McGraw, Sammy Migues and I have learned interviewing executives at some the world’s top software security programs. We’re in the process of building a maturity model out of the data we collected, but some of the stuff we learned is too good to wait. We wrote up a Letterman-style Top 10 Surprises article this month. Check it out. For me the biggest surprise was about penetration testing. Penetration testing is one of the first things many organizations do when it comes to software security. I believe it to be universally true that if you’re not paying attention to security, you have security problems, but it seems most people like to pay a penetration tester prove it to them. Now here’s the surprise: as organizations get better at software security, their emphasis on penetration testing diminishes. In most cases it doesn’t disappear, but it does get wrapped into a much larger and much more comprehensive approach to improving security. The best initiatives balance the yin and the yang of attack and defense. We passed an inflection point last year. People are now spending more money on getting code right than they are on proving it’s wrong. (More on that here.) This isn’t the end of the road for penetration testing, but it does change things. Rather than being a standalone “product”, it’s going to be more like a product feature. Penetration testing is going to cease being an end unto itself and reemerge as part of a more comprehensive security solution. This kind of thing happens all the time in high-tech. The first PC spell checkers were standalone programs, but the market for stand-alone spell checkers died when they became a standard part of any word processor. These days spell checkers are everywhere (even in my IDE), but there is no market for a standalone spell checker. Proof positive: there aren’t even any Web 2.0 or iPhone spell checker startups! Alright, so why 2009? The time is right because back in 2007, IBM bought a company named WatchFire and HP bought a company named SPI Dynamics. The acquired companies both made web application penetration testing products. IBM and HP spent serious money for these companies—not crazy dotcom prices, but even at HP and IBM you have to have to tell a good story before you get to spend north of seventy million dollars. The good story was that the acquired technology would work together with other products and services to fuel a broad entrée into a rapidly growing software security market. It takes a little while to digest any acquisition, but by now it’s been long enough. 2009 will be the year this strategy comes together, and when we look back, it will be the year when most of the world began thinking about penetration testing as part of a larger offering. There will always be boutique security consulting companies with funny names and exotic services, but the industry will grow by integrating security yin and yang. If you’d like a sneak preview of what the future holds, check out the work White Hat Security has done to integrate their vulnerability measurement service with Web application firewalls. This is attack and defense working together in a creative new way. More than ever before, people understand the software security challenge, and penetration testing deserves credit for helping spread the word. But knowing a security problem exists is not the same as knowing how to fix it. In other words, penetration testing is good for finding the problem and bad for finding the solution. Just like the venerable spell checker, it’s going to die and come back in a less distinct but more pervasive form.
Start State

Posted: October 30th, 2008, 7:18pm CDT by Brian Chess

Tags: [edit]

It's easy to visualize a world where you can actually trust the software you use. In fact, a lot of people try to pretend they live in that world right now! While the end state is easy to imagine, a lot of organizations have a lot more trouble figuring out how to exit the start state. In other words, what do you do first? How will you know if you're making progress? How do you know when you've arrived? We're going to try to answer some of those questions by creating a maturity model for software security. Based on a first draft created largely by Pravir Chandra, we've set up a software security framework. Check it out here. Now the data are starting to roll in. Gary McGraw, Sammy Migues, and I just finished our first round of maturity model interviews. We're talking to the people in charge of some of the most successful initiatives in the world and mapping what they've accomplished into the framework. We've got a huge amount of work in front of us, but the results are already fascinating.
Space Race

Posted: August 13th, 2008, 2:02am CDT by Brian Chess

Tags: [edit]

McGraw has published his roundup for the software security space in 2007. Lots of juicy numbers. Here it is. Mark 2007 on your calendar--it was a turning point. It was the first year there was a bigger market for products that help you get code right than there was for products that help you demonstrate a problem exists.
Secure code for the iPhone

Posted: March 14th, 2008, 5:49pm CDT by Brian Chess

Tags: [edit]

Seems like we security people don’t get to celebrate a victory nearly as often as we get to learn from defeat, so please take a moment to enjoy some good news. I’ll try to prolong the moment by saying this: I spend a lot of time telling programmers that they are no longer in a position where they can pretend that security isn’t part of their jobs, but my message is contravened by most of the training material programmers see. Whether it’s documentation a new framework, library, or platform, the standard play is to pretend that security doesn’t exist. No mention of what you need to do in order to make your code secure. No mention that security is anything other than a user name and a password. I’m happy to say that Apple is bucking this trend. Apple opened up the iPhone for third party applications last week. That means they’re providing programmers a software development kit and instructions on how to write code for the iPhone. On the home page for iPhone development, they have a list of common coding questions such as “How do I debug my application?” and “How can my application detect motion?” At the bottom of the list is “How do I write secure code?” The answer links to a secure coding guide that discusses topics like validating input and avoiding buffer overflow as well a discussion of the security services the iPhone’s OS provides. Nice. (In order to browse the links above, you have to register as an iPhone developer. It’s free. Go here.)
Bye-Bye Disk Encryption

Posted: February 22nd, 2008, 12:00am CST by Brian Chess

Tags: [edit]

Ed Felton and the gang at Princeton have struck again! This time they've figured out how to defeat the disk encryption schemes built into Vista, MacOS X, and others. The attack works because the values held in RAM don't disappear instantly when a computer is switched off. They decay slowly over a period of seconds, and that can be extended to minutes with a little bit of coaxing. That's long enough to boot up a second OS and read out the contents of memory. After that, it's just a matter of extracting the crypto keys, and the game is over. Awesome work. Read about Cold Boot attacks on Encryption Keys. If there's a lesson to be re-discovered here, I think it's the amazing way we end up building security systems on what seem to be solid ground (as in "computers forget stuff when you turn them off"), and only when it's too late do we find out that our premise was strong enough for trying to explain computers to someone like your old uncle Hugo, but not strong enough to adequately secure your data. It appears to me that this attack is still too sophisticated for the average thief who steals laptops in coffee shops, but it's plenty easy for the forensics guy down at the police station.
The New (De)face of Cybercrime

Posted: February 19th, 2008, 1:16pm CST by Brian Chess

Tags: [edit]

Some inspired soul has reworked the New Face of Cybercrime trailer. Watch this. My favorite bit: the cat and the baby.
WTFs/Minute

Posted: February 6th, 2008, 12:33pm CST by Brian Chess

Tags: [edit]

We don't get that many cartoons about code review, so when a new one crops up, it's cause for celebration. If you don't think it's hilarious, you need to do more code review. Many thanks to Mark Curphy.
SATE

Posted: February 5th, 2008, 1:36am CST by Brian Chess

Tags: [edit]

NIST has announced what they call a Static Analysis Tools Exposition (SATE). Participating static analysis tools will be used to analyze a small set of programs selected by NIST. Eventually all of the results will be made public. I like the idea a lot, and I'm going to make sure Fortify takes part in both the C and Java tracks. Running a bunch of programs against the same test cases might seem like a straightforward exercise, but it's not. Finding an even-handed way to examine complicated tools is tough, and the fact that NIST is emphasizing the detection of security vulnerabilities makes it that much tougher. (Tell me again, what exactly constitutes a "vulnerability" in source code?) Just browse through the steps in the SATE protocol to get a feel for the complexity involved. To make the exercise even more difficult, everyone who makes a tool can't help but worry that they'll come out looking bad. After all, this sounds a little bit like the setup for a product review. We've seen those before. (Here's a link to the one Network Computing did in the middle of 2007: link.) Product reviews generally have winners and losers. NIST says they'll work hard to avoid these kinds of labels, but it's not going to be easy. All of this makes test case selection a touchy subject, and comparative presentation of the results is an absolute minefield. The problems are many, but there's also a lot to be gained. Here are the three reasons I'm looking past all of the complexity and anxiety and participating in SATE: 1) People who want to run their own tool comparisons will benefit. You might like the way SATE works or you might not, but you'll be able to see what they did and where it took them. 2) Tool builders will benefit. Static analysis for security is a hot topic in both industry and academia. The results will give us a little bit of insight into which problems we've conquered and which ones require more work. 3) The security community will benefit. Being able to compare output from different tools will give us a better idea about how the tools classify code, and my guess is that we'll learn that our nomenclature still needs improvement. Better nomenclature makes it easier to talk to the 99% of humanity that doesn't think about software security every day.
They Set the Wii Free

Posted: January 30th, 2008, 1:49am CST by Brian Chess

Tags: [edit]
It appears that a buffer overflow vulnerability in The Legend of Zelda for the Wii, and that’s all hackers need in order to run their own code on the machine. Here’s the scoop. Pretty soon this will mean game pirating for the Wii without the need to get out your soldering iron. That’s important if you want to get people to steal games in the same casual manner they steal music. Piracy is a big deal for all modern gaming platforms because most of the money comes from software (games) not from hardware. Apple has run into similar trouble recently. It seems that more than a million iPhones have been unlocked, and I think it’s safe to assume that it’s mostly happening using the simplest mechanism out there, a web site that exploits a buffer overflow in the iPhone. That means Apple is missing out on their cut of the phone subscription going to AT&T, and that has to hurt. The moral to this story is this: if your business relies on the security of your software, you’ve got two choices:
- Design a system that will be secure even when people discover a few bugs.
- Make sure you don’t have even a single bug.
Neither of those options is easy, but creating a robust design not nearly as hard as creating a flawless implementation.
Analyzing the Analyzers: Looking at Source Code for Breathalyzers

Posted: January 28th, 2008, 1:25pm CST by Brian Chess

Tags: [edit]
For as long as there have been breathalyzer machines, DUI suspects have been looking for creative ways to beat them (see newspaper clipping below.) The latest trend is to go after the source code. Here are three recent cases:
- August 2007: Minnesota Supreme Court rules that source code must be revealed.
- December 2007: Source code analysis in New Jersey class-action breathalyzer case.
- January 2008: Kentucky appeals court rules that source code must be revealed.
My favorite anecdote so far comes from the New Jersey analysis. One of the teams used Fortify to analyze the code, and lo-and-behold, they found a buffer overflow vulnerability! This raises the possibility that if you mix just the right cocktail at just the right time, you could craft an exploit. (Dream on.) The real lesson here is that our legal system is waking up to the importance of code. If the code isn’t trustworthy, the outcome isn’t trustworthy either. (Electronic voting machine vendors, you might want to read that last line again.) If the code provides evidence that the programmers weren't being careful, that's going to be bad news for the vendor.
The Checklist

Posted: January 25th, 2008, 8:54am CST by Brian Chess

Tags: [edit]

“What do you do when expertise is not enough?” Atul Gawande Hundreds of little decisions per day. Screw up any one of them, and disaster could ensue. Smart and highly trained professionals who know what to do, but who aren’t always focused on the details. A transition from a cowboy mentality to an emphasis on a boring but safe and repeatable process. Sound like the hymn of software security? It does indeed, but it’s also the way Atul Gawande describes modern intensive care units in his article The Checklist. Intensive care units appear to suffer from many of the same problems the software community faces. Too much complexity creates too many opportunities for error. Little mistakes lead to complications, and those complications can, quite literally, kill. As with software, there is no easy or complete answer to the problems doctors face in the ICU, but the surprising finding this article points out is that a simple tool, the checklist, is enormously effective. Arm nurses with a five step checklist for avoiding an infection when inserting a line, and secondary infection rates plummet. Do programmers in your organization have the equivalent for common security-sensitive operations? Maybe those cross-site scripting errors keep popping up because there’s no template to guide programmers as they create new code. If you’re still casting around for one or two more new year’s resolutions, resolve to give programmers good examples they can follow. In the book, we try to give a positive, correct, and secure code listing for every broken, bad, or vulnerable piece of code we discuss.
Getting Started

Posted: January 14th, 2008, 2:52pm CST by Brian Chess

Tags: [edit]

Software security can be hard to talk about, sometimes because people who make software are an incredibly diverse bunch. I’m not talking about globalization, I’m referring to the fact that the term “software” covers a lot of ground. Building operating systems, web sites, cell phones, and airplanes all require major software chops, and they all require an understanding of security, but they are very different undertakings. If you’re talking to someone about the new software security initiative you just got off the ground, the result can be wild confusion before you figure out that the other person’s take on what a “software security initiative” means is completely different than your own. I like McGraw’s latest DarkReading column because it gives a nice overview of the different ways people get started.
Web Vulnerabilities in the Age of the iPhone

Posted: July 14th, 2007, 9:03pm CDT by Brian Chess

Tags: [edit]

I've read lots of articles about hacking the iPhone lately. They're mostly focused on exploring the hardware, circumventing the intended activation process, or putting new software on the device. That's nice, and I like to see a technological marvel smashed open with a hammer just as much as the next geek, but I'm more interested in how the iPhone changes the balance of power when it comes to security. Small items first: There are a few minor application issues that make a phisher's job easier. For instance, the email client does not display the URL you're going to visit when you click on a link, so a phisher can send out spam that says Click here to go to PayPal and point the link to http://paypal.fakesite.ro, and there's no way to know what's going to happen until you click. Once you do click, the browser displays only the first 20 or so characters of the URL, so its easy to hide a big gnarly cross-site scripting attack without arousing any suspicion. Alternatively, the phishing site can use JavaScript to scroll the URL bar out of site. (See Joe Hewitt's writeup.) I expect issues like these will be addressed soon enough, just as they've been with all of the major desktop mail clients and Web browsers. Much more interesting is the way the iPhone connects the Web browser and the phone. As the author of a Web site, you can embed a telephone number in a web page like this: <a id="phone_home" href="tel:1-900-867-5309">call me!</a> You can also write JavaScript that causes the iPhone to initiate the dialing process: <script> window.document.url = "tel:1-900-867-5309" </script> When that code runs, the user will be prompted "1-900-867-5309 (call) (cancel)". If the user accepts, the phone dials. Now you can turn phishing into money faster than ever before because the payload is the product: victims dial a 900 number, and the money starts rolling in. By the way, setting up a 900 number is easy. Alternatively, use a cross-site scripting vulnerability to have a banking Web site initiate a call to a fake technical support number. What's the first thing the fake support rep asks for? Your account information of course! After all, you called them, so they need to "confirm your identity". Once again, an old scam gets new legs with a little help from the latest technology, and once again the ante on cross-site scripting goes up. I expect two things will happen in coming year: 1) We'll learn about more cute tricks that web applications can use to look more like native iPhone applications and to interface with the iPhone and allow access to things like contacts, photos, and maybe even the phone's physical location. All of these features will expand the horizons of enterprising attackers. 2) All of the other handset makers in the world will begin to deliver their response to the iPhone. At that point, they will all have been working around the clock in panic mode for the better part of a year, and the devices will contain a treasure trove of security vulnerabilities that make the iPhone look like Fort Knox. After all, Apple got plenty of things right: at least you have to confirm before the phone dials.
Exploiting Online Games

Posted: July 13th, 2007, 1:37am CDT by Brian Chess

Tags: [edit]

Hoglund and McGraw's new book just came out. It's brilliant. Exploiting Online Games: Cheating Massively Distributed Systems is a peek into the future and an amazing recruiting tool all rolled into one. If you already understand the kinds of security problems today's software creates, then the book gives a view into the kinds of security problems tomorrow's software is going to create. Here's a hint: Time and State. Massive distributed systems can fail in the same ways that centralized (traditional) systems can fail, but they're also likely to run trouble as they try to coordinate critical data between nodes. There is a lot of money flowing through modern games, and so any sort of defect fuels the fight between the game companies and people who seek to profit from them. But if you just think games are cool and want to know more about what your options are for getting ahead, or if you're trying to figure out why your arch-nemesis from the next dorm building seems to have such good aim with his Crossbow of Ultimate Peril, this book has all sorts of tricks, hacks, code examples, and step-by-step information on how to get inside the mother of all massive multiplayer online games: World of Warcraft (WoW). When I was in college, there was a whole subculture of people who learned to program so that they could extend and improve Multi-User Dungeons (MUDs), an early forerunner of WoW. This book is going to be the gateway to programming for the WoW generation. And, if everything goes as I'm sure Hoglund and McGraw have planned, those kids won't even know its possible to program without thinking about security.
Secure Programming with Static Analysis

Posted: July 5th, 2007, 2:23am CDT by Brian Chess

Tags: [edit]
About 18 months ago Jacob West and I got serious about writing a book. With much joy and relief, I'm happy to say that the book is finally out. Secure Programming with Static Analysis has been released into the wild! In some ways this is an extended-play version of my PhD work, but written for more than just an academic audience. It makes the case for static source code analysis as an essential tool for getting software security right. The book covers a lot of ground.
- It explains why static source code analysis is a critical part of a secure development process.
- It shows how static analysis tools work, what makes one tool better than another, and how to integrate static analysis into the SDLC.
- It details a tremendous number of vulnerability categories, using real-world examples from programs such as Sendmail, Tomcat, Adobe Acrobat, Mac OSX, and dozens of others.
- It includes tutorials that walk through applying a static analysis tool to a real programs.
I'm now going to face the happy problem of finding something to do with my weekends! I can handle it. I'll send a free book to the first person who names the type of flower on the cover and gives a decent explanation for why we used a flower rather than a lock or a gate or some kind of weapon. Post your answer here or send me email (my first name @ fortify.com). Update: The prize goes to Andre Gironda who named the flower as a lotus, and who correctly guessed that we used a lotus because it is a symbol of purity and enlightenment. It begins in the mud, but transforms itself into a beautiful flower. We like the idea that static analysis helps bring purity, and that taking security into account is part of an enlightened approach to programming. We also like the way it looks.
Sorry Apple, Wrong Answer

Posted: June 12th, 2007, 1:59am CDT by Brian Chess

Tags: [edit]

I love my iPod. I love my Powerbook. Fake Steve Jobs is not too shabby. Along with the rest of the universe, I was preparing to love the iPhone, until today when the real Steve Jobs announced the way third party developers would be allowed to add applications: the Web. That's right, the iPhone will come complete with Safari, and developers will be able to harness the power of Web 2.0 to create software with all of the same bells and whistles as native iPhone applications. According to Apple, this allows third party applications to "extend the iPhone's capabilities without compromising its security or reliability." Uh ... that's scary. It's already hard to get Web security right, and giving the Web browser access to your contacts, your photos, and your music just ups the penalty for getting it wrong. If the Web is the platform of the future, then cross-site scripting is the next buffer overflow. This is bad news. I'm going to stop writing iPhone and start writing iP0wn.
Targeting the Black Box

Posted: April 25th, 2007, 1:27pm CDT by Brian Chess

Tags: [edit]

Black box testing is software testing. It's time for black box testing to start applying some of the same discipline used in the software testing world. Most organizations admit they don't know how thorough their black box testing is, but most people assume their tests are comprehensive and effective. More than half of our survey participants believe their testing covered at least 60% of their applications, but the data suggest they're not doing as well as they think. We looked at two of the leading black box tools as they probed five small web applications. Our data suggests that even for small applications, people using black box tools are not even getting 30% coverage. (Looking at larger apps generally makes the situation even worse.) Applying manual effort to customize the tools can significantly improve their effectiveness, but the tests still failed to achieve coverage numbers greater than 50%. Conclusion: black box tools aren't a good a substitute for good testing techniques. In the past, people have used black box testing as a way to attract attention to the security problem. For that purpose, coverage doesn't matter: all you have to do is find some security holes. Software security is starting to mature a little bit, and now we need to start looking at whether or not our security testing is effective. We can borrow a lot from software testing. It's time for security testing to start growing up!
JavaScript Hijacking: Who's Responsible?

Posted: April 2nd, 2007, 1:42am CDT by Brian Chess

Tags: [edit]

We just released our report on the first (and to our knowledge the only) type of attack that specifically targets Ajax-style web applications. The attack is called JavaScript Hijacking. The report is here. As part of the work, we took a look at 12 Ajax frameworks, including Google's GWT, Microsoft Atlas, Yahoo! UI, and a number of open source projects. A lot of the open source projects provide only client-side JavaScript libraries. In the report we point out that almost none of the frameworks protect against JavaScript Hijacking or give programmers any indication that there's anything they need to protect against themselves. Thus-far, nobody has questioned us on the technical aspects of the attack, and quite a few of the framework maintainers have said they plan to address the problem. But we've taken some flak for our proposed solutions from a few owners of client-side JavaScript libraries. It boils down to one sentence in the report: "Preventing JavaScript Hijacking requires a secure server-side implementation, but it is incumbent upon the client-side libraries to promote good security practices." Some of the client-side guys aren't so happy with that. They say that security is totally a server-side problem. Keep in mind, most of these projects don't say a thing about writing a secure server in their documentation, and worse yet, some of them actually require the server to be vulnerable unless the application programmer wants to start monkeying around in the guts of the framework. Ugh. Now that I'm done writing the report, I can ditch the any semblance of an even or balanced tone, so let me try saying it again. Here's the deal: if you're going to write some code that you expect programmers to re-use, and if that code walks people right into creating a security cesspool, you are doing the world a disservice. Don't be surprised when I tell the world to steer clear of your stuff.

My Security Planet » Items by Brian Chess

Feeds

Items by Brian Chess

Posted: December 15th, 2008, 11:36pm CST by Brian Chess

Tags: [edit]

Posted: October 30th, 2008, 7:18pm CDT by Brian Chess

Tags: [edit]

Posted: August 13th, 2008, 2:02am CDT by Brian Chess

Tags: [edit]

Posted: March 14th, 2008, 5:49pm CDT by Brian Chess

Tags: [edit]

Posted: February 22nd, 2008, 12:00am CST by Brian Chess

Tags: [edit]

Posted: February 19th, 2008, 1:16pm CST by Brian Chess

Tags: [edit]

Posted: February 6th, 2008, 12:33pm CST by Brian Chess

Tags: [edit]

Posted: February 5th, 2008, 1:36am CST by Brian Chess

Tags: [edit]

Posted: January 30th, 2008, 1:49am CST by Brian Chess

Tags: [edit]

Posted: January 28th, 2008, 1:25pm CST by Brian Chess

Tags: [edit]

Posted: January 25th, 2008, 8:54am CST by Brian Chess

Tags: [edit]

Posted: January 14th, 2008, 2:52pm CST by Brian Chess

Tags: [edit]

Posted: July 14th, 2007, 9:03pm CDT by Brian Chess

Tags: [edit]

Posted: July 13th, 2007, 1:37am CDT by Brian Chess

Tags: [edit]

Posted: July 5th, 2007, 2:23am CDT by Brian Chess

Tags: [edit]

Posted: June 12th, 2007, 1:59am CDT by Brian Chess

Tags: [edit]

Posted: April 25th, 2007, 1:27pm CDT by Brian Chess

Tags: [edit]

Posted: April 2nd, 2007, 1:42am CDT by Brian Chess

Tags: [edit]