Every good things have an end... this is the time for me to leave NIST. So I will be a security consultant at Cigital, Inc..

I've been working at NIST for 2 years and a half as a Guest Researcher in the SAMATE Project. I originally came at NIST to do mostly statistical analysis or so, but it changed a lot! I started by building the SAMATE Reference Dataset website and this is how I started to learn about "security", but working with flawed source code. This was very obscure to me (I guess like everybody computer scientist specialized in applied mathematics) and I learned a lot about weaknesses, vulnerabilities, "how to find them?", scanners etc.

My first real security related work was about the Web Application Security Scanner Specification and then, design a way of testing the web apps scanners:

• test suite with seeded vulnerabilities
• checking the types of attacks
• trying to explain the false-negative of the tools by a monitoring of what/where the scanner went in the application at a logical level, such as "did the tool logged in successfully? did it generate a couple of errors, did it try many times?

The goal of the 3 components based analysis is to really be able to understand what the tool is doing, if it didn't find a particular vulnerability, why?

One of the best moments I had at NIST was when we did the Static Analysis Tool Exposition. I was part of the organizers and from the beginning, it was a real challenge: choosing good test cases, criteria to evaluate the reports, etc. Of course, SATE 2008 was not perfect, we did many mistakes, but at least, we tried, we had some results and we learned a lot. I have good hopes for the next SATE, even though this is really challenging on many aspects:

1. Not make people think/act like this is a competition (we sometimes see people claiming they won SATE 2008, but... well, there would be many things to say to them)
2. Having a strong evaluation criteria (I guess this is challenging every time human assessment is part of the game)
3. Solve the way to present data to the evaluators. We couldn't have the GUI of the tools etc. so our analysis (as an evaluator) was really limited and we sometimes had to guess what was the exact weakness report
4. and finally, having more resources and help for evaluating the weaknesses reported by the tools (47k this year, one month to evaluate...)

Oh well, I will of course continue to follow what the SAMATE team is doing, even though I will be away and busy with other interesting stuff and I'm really looking forward to see the results of the current study we are running on the function-wise weakness characterization.

But for now, it's time for me to get some vacation, going back to France for almost one month, getting my worker visa etc.