deep inside: security and tools

NIST Static Analysis Tool Exposition special publication released

“The NIST SAMATE project conducted the first Static Analysis Tool Exposition (SATE) in 2008 to advance research in static analysis tools thatfind security defects in source code. The main goals of SATE were to enable empirical research based on large test sets and to encourage improvement andspeed adoption of tools. The exposition was planned to be an annualevent.”

SATE 2008 was one of my last project at NIST. I really enjoyed working on this project from the beginning, it was challenging especially because we hadto create so many artifacts to make the tool reporting the weaknesses the sameway, integrate them all together and provide ways for assessors to makemeaningful reviews.

In a nutshell, we selected 6 different open-source programs (3 in C, 3 in Java) and made tool vendors running their tool on these test cases. Tool vendors were allowed to customize their tool if their tool provide such capability. Fortify was the only vendor who created a custom rule (to help the tool with a validation routine for MVNForum). Our goal was then to combine theresults all together and analyze: provide information on the correctness of thetool.

If you are interested, you can download the SATE data andthe NIST SATE Special Publication.

Thanks to all the SAMATE team for this effort, and especially Vadim Okun andPaul E. Black.

For more information, you can reach the SATE page at NIST.

All entries

  1. February 2013 — RSA 2013 speaking session
  2. February 2013 — HTML5 tokenization visualization
  3. September 2011 — PHP, Variable variables, Oh my!
  4. July 2011 — Dissection of a SQL injection challenge
  5. January 2010 — Yes, we need a standard to evaluate SAST, but it ain't easy...
  6. November 2009 — Data driven factory: I give you data, you give me an object...
  7. June 2009 — NIST Static Analysis Tool Exposition special publication released
  8. December 2008 — Every-day's CSRF: Sorry, I turned off your christmas tree lights
  9. August 2008 — Why the "line of code" is indeed a good metric
  10. May 2008 — Accelerate the convergence to the bug: Running the test in 16-bit
  11. February 2008 — Code review tools: the missing link (so far)
  12. January 2008 — Talk: Problems and solutions for testing web application security scanners
  13. October 2007 — IE6 And IE7 don't have compatible CSS tricks
  14. September 2007 — Source Code Obfuscation
  15. February 2007 — The return of the SVG XSS
  16. February 2007 — How you should design a test suite for Web Apps Scanners
  17. January 2007 — Test Suites for Web Application Scanners
  18. December 2006 — SVG Files: XSS attacks