deep inside: security and tools

Code review tools: the missing link (so far)

First of all, I do not consider myself as a pen-tester so maybe you will find these ideas irrelevant, stupid or useless... I have been doing some pen-testing though, whether it was for some friends, for fun (yeah, it's good to learn like that) or for profit (well, it was kinda part of my job for SATE 2008) so I'm not that n00b but I am not a pen-tester. I am not an expert in pen-testing and code review. But when I do some, at work, I have the chance to be able to use commercial tools — I say it's a chance because there is a real benefit of using such tools. In fact, tools are good, way better than me, they can find thousands of vulnerabilities in minutes... I cannot; I need way more time. But here is a little feedback vendors can have from me, utilizing the tools.

The tools are amazing to find some defects, saying that something doesn't look good to them and giving you a stack of 42 function calls. Eh! that's part of the job to examine this bunch of function in order to see why the tool reported this as a vulnerability.So, examining the functions means looking how the data will be transformed/transported from a point to one another. And I cannot tell you the pain it is to do that for the dozens of reported vulnerabilities where the correctness of the tool is not obvious (at least for me).

While talking about that with Vadim today, I thought of a tool that would be awesome for a code reviewer in order to facilitate the “correctness tests”. The idea is really simple and maybe the tool already exists — if so, please give me a link! — but what if you had a kinda debugger where you were able to select the point where you want to start the dynamic evaluation of a piece of code (the Entry Point) and the point where you want to finish and see the result (the Break Point). What is the difference with a typical debugger? The possibility to do such in relation with the source code. In the interface of the source code analyzer, I would be able to select the entry point I want to start my dynamic analysis and the break point. I would launch the dynamic evaluation which would go to the state of the entry point (maybe by asking how to go there... there is often multiple paths to go to one branch of the code), then I would do the modification I want (trying to bypass some filters for example with some weird strings) and the dynamic engine would run the piece of code until the Break Point; then look at the result.

What I just described is a really narrow view of such combination of static/dynamic analysis, by doing a step-by-step modification of the values. We could have information of the privilege state of the current user for a web application, would be able to replay easily a la web apps scanners, etc.

I know that building such a tool is doable. Hard but definitely doable. So far, the toughest point I saw is to be able to arrive at a given state of the program. You would need to do a binary coverage and looking at the branches to take, recording these and mapping the records with the source code. Once you're done with it, you're ready for modifying the parameters, and to look at the results. Yes, the main difference with a debugger is to come in a given state referenced by a function call. But wouldn't this help you to figure out the correctness of a given piece of code?

All entries

  1. February 2013 — RSA 2013 speaking session
  2. February 2013 — HTML5 tokenization visualization
  3. September 2011 — PHP, Variable variables, Oh my!
  4. July 2011 — Dissection of a SQL injection challenge
  5. January 2010 — Yes, we need a standard to evaluate SAST, but it ain't easy...
  6. November 2009 — Data driven factory: I give you data, you give me an object...
  7. June 2009 — NIST Static Analysis Tool Exposition special publication released
  8. December 2008 — Every-day's CSRF: Sorry, I turned off your christmas tree lights
  9. August 2008 — Why the "line of code" is indeed a good metric
  10. May 2008 — Accelerate the convergence to the bug: Running the test in 16-bit
  11. February 2008 — Code review tools: the missing link (so far)
  12. January 2008 — Talk: Problems and solutions for testing web application security scanners
  13. October 2007 — IE6 And IE7 don't have compatible CSS tricks
  14. September 2007 — Source Code Obfuscation
  15. February 2007 — The return of the SVG XSS
  16. February 2007 — How you should design a test suite for Web Apps Scanners
  17. January 2007 — Test Suites for Web Application Scanners
  18. December 2006 — SVG Files: XSS attacks