Every good things have an end... this is the time for me to leave NIST. So I
will be a security consultant at Cigital,
Inc..
I've been working at NIST for 2 years and a half as a Guest Researcher in
the SAMATE Project. I originally came at
NIST to do mostly statistical analysis or so, but it changed a lot! I started
by building the SAMATE Reference
Dataset website and this is how I started to learn about "security", but
working with flawed source code. This was very obscure to me (I guess like
everybody computer scientist specialized in applied mathematics) and I learned
a lot about weaknesses, vulnerabilities, "how to find them?", scanners etc.
My first real security related work was about the Web Application
Security Scanner Specification and then, design a way of testing the web
apps scanners:
- test suite with seeded vulnerabilities
- checking the types of attacks
- trying to explain the false-negative of the tools by a monitoring of
what/where the scanner went in the application at a logical level, such as "did
the tool logged in successfully? did it generate a couple of errors, did it try
many times?
The goal of the 3 components based analysis is to really be able to
understand what the tool is doing, if it didn't find a particular
vulnerability, why?
One of the best moments I had at NIST was when we did the Static Analysis Tool Exposition. I
was part of the organizers and from the beginning, it was a real challenge:
choosing good test cases, criteria to evaluate the reports, etc. Of course,
SATE 2008 was not perfect, we did many mistakes, but at least, we tried, we had
some results and we learned a lot. I have good hopes for the next SATE, even
though this is really challenging on many aspects:
- Not make people think/act like this is a competition (we sometimes see
people claiming they won SATE 2008, but... well, there would be many things to
say to them)
- Having a strong evaluation criteria (I guess this is challenging every time
human assessment is part of the game)
- Solve the way to present data to the evaluators. We couldn't have the GUI
of the tools etc. so our analysis (as an evaluator) was really limited and we
sometimes had to guess what was the exact weakness report
- and finally, having more resources and help for evaluating the weaknesses
reported by the tools (47k this year, one month to evaluate...)
Oh well, I will of course continue to follow what the SAMATE team is doing,
even though I will be away and busy with other interesting stuff and I'm really
looking forward to see the results of the current study we are running on the
function-wise weakness characterization.
But for now, it's time for me to get some vacation, going back to France for
almost one month, getting my worker visa etc.
Last comments