Tuesday, June 30 2009
By Romain on Tuesday, June 30 2009, 11:30 UTC
The NIST SAMATE project conducted the first Static Analysis Tool
Exposition (SATE) in 2008 to advance research in static analysis tools that
find security defects in source code. The main goals of SATE were to enable
empirical research based on large test sets and to encourage improvement and
speed adoption of tools. The exposition was planned to be an annual
event.
SATE 2008 was one of my last project at NIST. I really enjoyed working on
this project from the beginning, it was challenging especially because we had
to create so many artifacts to make the tool reporting the weaknesses the same
way, integrate them all together and provide ways for assessors to make
meaningful reviews.
In a nutshell, we selected 6 different open-source programs (3 en C, 3 in
Java) and made tool vendors running their tool on these test cases. Tool
vendors were allowed to customize their tool if their tool provide such
capability. Fortify was the only vendor who created a custom rule (to help the
tool with a validation routine for MVNForum). Our goal was then to combine the
results all together and analyze: provide information on the correctness of the
tool.
If you are interested, you can download the SATE data and
the NIST SATE
Special Publication.
Thanks to all the SAMATE team for this effort, and especially Vadim Okun and
Paul E. Black.
For more information, you can reach the SATE page at NIST.
Saturday, February 21 2009
By Romain on Saturday, February 21 2009, 15:11 UTC
Fortify just posted a nice blog
post about the audit they did on several reference implementation that
compete for being the next NIST SHA-3.
They do not release much information on their findings: only one is
described. I would have really like to see how powerful was the analysis (if it
was) to find these problems.
It could be nice too to see other tool vendors, such as Grammatech,
Klocwork, Coverity, etc. to do the same, and then, start another competition
;)
I'd really like to emphasize the conclusions in the Fortify's blog post:
Reference implementations don't disappear, they serve as a starting point
for future implementations or are used directly. A bug in the RSA reference
implementation was responsible for vulnerabilities in OpenSSL and two seperate
SSH implementations. They can also be used to design hardware implementations,
using buffer sizes to decide how much silicon should be used.
The other consideration is speed, which will be a factor in the choice of
algorithm. The fix for the MD6 buffer issues was to double the size of a
buffer, which could degrade the performance. On the other hand, memory leaks
could slow an implementation. A correct implementation is an accurate
implementation.
Thursday, September 25 2008
By Romain on Thursday, September 25 2008, 09:01 UTC
I know how tough and crucial it is to get participants to a survey, so that
would be great if you guys could take this and spread it a little bit
more...
Researchers at ThePrivacyPlace.Org are conducting an online survey about
privacy policies and user values. The survey is supported by an NSF ITR grant
(National Science Foundation Information Technology Research) and was first
offered in 2002. We are offering the survey again in 2008 to reveal how user
values have changed over the intervening years. The survey results will help
organizations ensure their website privacy practices are aligned with current
consumer values.
The URL is: http://theprivacyplace.org/currentsurvey
We need to attract several thousand respondents, and would be most
appreciative if you would consider helping us get the word out about the
survey, which takes about 5 to 10 minutes to complete. The results will be made
available via our project website (http://www.theprivacyplace.org/).
Prizes include $100 Amazon.com gift certificates sponsored by Intel Co. and
gifts from IBM and Blue Cross and Blue Shield of North Carolina
On behalf of the research staff at ThePrivacyPlace.Org, thank you!
Friday, February 29 2008
By Romain on Friday, February 29 2008, 22:55 UTC
This evening at work, with Vadim, we were exhausted after days of work but
we were smiling. Smiling and happy because we knew that the step 3 of SATE was pretty much
done. The step 3 is when all the participants are sending their output
to us. Even if we know that we will have hard time to come up with the master
reference list for each test cases what we selected for SATE 2008, we know that
this is interesting data for the SwA community and especially SCA studies.
Today, we can finally tell which test cases were selected by us for SATE
2008. First of all, we have 2 different tracks: C language and Java language.
For the java track, we decided to look more into web applications. We then
have:
And for the C track we selected:
- Nagios: host, service and
network monitoring with web interface (using CGI)
- Lighttpd: web
server
- Naim: console instant
messenger
You may have lots of comments on why these and I am totally ready to answer
your questions. Just to let you know, during the selection phase, we reviewed
50+ different applications. For each applications, we had to scan them using
tools, doing some manual review and our main goal is to find at least one
exploitable vulnerability. Concerning the type of test cases themselves, the
constrain is to have real exploitable vulnerabilities and they must be real
applications which means basically, not test cases that we have in our SRD.
Just as reminder, the next important dates for SATE 2008 are:
- April 15, we are distributing to the participants our master reference
list, the list of real weaknesses found by the
participants
- June, comparison of all the participants results, the participants get all
the reports submitted at SATE 2008
- December, all the data and reports are public
Thursday, February 21 2008
By Romain on Thursday, February 21 2008, 09:28 UTC
Just to say that I am please to see the OWASP Chapter France starting again thanks to Sebastien Gioria! I hope that this is gonna last for good and that we will be able to spread the web security & tools in France. Even though I am not in France anymore, I am please to be part of the board. What I would like to do so far, is to talk with engineering school, universities, etc in order to make web security as part of classes when students are learning about web development for instance (or just development).
In the same time, we are releasing the translation of the OWASP Top Ten 2007 in French. The document by itself is a really good content! The French translation has been done while trying to keep the original ideas of the Top Ten.
You can download the OWASP Top Ten 2007 in French on the OWASP Chapter France web page. As usual, every comments, ideas etc, about the role of OWASP in France are more than welcome!
Thursday, February 14 2008
By Romain on Thursday, February 14 2008, 19:14 UTC
Tomorrow will start SATE 2008: the registered participants will be able to get the test cases associated to the tracks they want to participate in. They will have until the 29th of February to send the report of the tools. We are all pretty excited here before the start. It was a real rush for finding the test cases that we think are good for such an event...
Anyway, just a news to release a python script which is definitely SATE oriented. The idea is only to convert the output of some free tools into the SATE XML format. The script is handling Flawfinder, ITS4 and RATS. It can also look at the NVD for the product and the version in order to retrieve the known vulnerabilities.
You can download the script weaknesses walker as a zip file or just the python script (you will need wwwCall for the NVD scrapping part; wwwCall is also included in the zip).
Example how to use ww with flawfinder:
./ww.py --tool flawfinder --file myproject.out.xml --format sate /home/romain/myproject
or for the NVD scrapper:
./ww.py --vdb winamp 5.2 --file winamp_5.2.nvd.xml
For the next version of ww, I may add the possiblity to play with the SATE XML format itself, such as merging the results of different tools with comparison of report or even just the report of multiple tools...
Also, if you are coming downtown DC this weekend for ShmooCon or even BlackHat DC 2008, if you wanna have a beer just drop me a mail. I wasn't able to find a ticket for Shmoo so will not go, but I will meet with dre and marcin from ts/sci security... so if you are around, just tell me I would be happy to meet more sec. people
The last thing is that this post is my number 100!
Tuesday, February 5 2008
By Romain on Tuesday, February 5 2008, 08:18 UTC
I've was happy yesterday when I learned that Fortify will participate to the Static Analysis Tool Exposition (SATE) we are currently organizing. And even more when I saw this morning Brian Chess blogging about SATE.
We've been working on SATE since our last Static Analysis Summit and, helped with a couple of existing exposition already existing at NIST such as TREC etc. for the guidelines, the rules and so on.
But even so, we had some example, we had three difficult tasks:
- Make people agree on the fact that it is not a competition
- Make vendors participating (if you are a vendor, reading this please, subscribe for participating at SATE)
- Choosing the test cases
The last point is not solved yet, and even, none of them can be considered as solved since not everybody is participating to the 2008 exposition (which has 2 tracks: C and Java), but we've been seeking for good test cases in C and Java. Good test cases... means not too big, not too small and having exploitable vulnerabilities. By the way, if any of the readers of this blog have some idea of Java or C test cases that would be good test cases, please, send me links, ideas or whatever :)
Anyway, SATE is on his way, I hope more tool makers will sign up for participating at this experiment.
Maybe another point, due to my usual blogging on web security and web apps security scanners, if SATE is a success as we expect it to be, we may open new tracks for... web application security scanners and I would love to have special tracks for security metrics (I want to show up!! :p)
Monday, January 28 2008
By Romain on Monday, January 28 2008, 10:21 UTC
While surfing the web, I found this website: http://opensourcetesting.org/.
Just the perfect repository of testing tools, there are a bunch of them on different testing area (security, functional, quality, unit testing and so on!).
Edit: Added in my security planet!
Monday, December 10 2007
By Romain on Monday, December 10 2007, 11:14 UTC
The Web Application Security Consortium is pleased to announce the first results of the
Script Mapping project! At this stage in the project we were able to cover most of the
test cases for Internet Explorer 7, Firefox 2 and Safari 3.
The results can be found on the project page:
http://www.webappsec.org/projects/scriptmapping/
Project Description:
The purpose of the Script Mapping Project is to come up with an
exhaustive list of vectors to execute script within a web page without
the explicit use of <script> tags. This data can be useful when testing
poorly implemented Cross-site Scripting blacklist filters, for those wishing
to build an html white list system, as well as other uses.
WASC is actively seeking volunteers from various sections of the
community including penetration testers, security researchers,
and developers to contribute to this project.
If you would like to be involved with the project or if you have comments about the
results, test cases etc., please contact me.
Saturday, December 8 2007
By Romain on Saturday, December 8 2007, 19:43 UTC
I love iGoogle. I have a couple of widgets and my RSS feeds in it... That's actually the problem, I had too much feeds, so I decided to create my own "planet" in order to have just one feed with all security blogs I'm reading.
You can reach it here if you have the same taste as mine: http://rgaucher.info/planet
I know that planet-security is pretty much doing the same, but I don't like the interface of this website and it doesn't have all the feeds I'm following...
Wednesday, November 21 2007
By Romain on Wednesday, November 21 2007, 19:32 UTC
Grabber was a nice project. The main goal for me was to learn stuff around web application security/scanners; I didn't really know much before I started this project.
But now that I've been playing with web apps scanners for more than 10months, I need to create a new one and go deeper in heuristics, browser integration and AI.
Grabber was in fact more a spider+fuzzer than something else... Not a good web apps scanner at all. Thinking of the analysis engine... It's something kinda stupid, no JavaScript execution, just simple heuristics for parsing and levenstein distances ;)
Anyway, I decided to start over this project. It's not gonna be a bunch of python scripts anymore, I am gonna use Qt/C++ extensively. The idea if this project is to be pen-testers oriented and open, I want to create a kind of wrapper around WebKit (especially using QtWebKit), a spider as core utilities and after, using plugins. The plugins should be either in C++ or JavaScript (QtScript actually). So far, we are 3 guys thinking of this project: we didn't start yet but we are open to every contribution; the project will of course be free and GPL'd.
I just post this in order to get some comments or suggestions about what a web apps scanner should do... Feel free to comment/mail...
Tuesday, October 16 2007
By Romain on Tuesday, October 16 2007, 10:26 UTC
I'm not usually posting news about article, blog entries etc. But this one is actually a very good one.
Michael Howard is actually sharing his 5 years experience about secure code.
Here is the article: http://msdn.microsoft.com/msdnmag/issues/07/11/Lessons/
Thursday, August 23 2007
By Romain on Thursday, August 23 2007, 15:36 UTC
Here is a new interesting project: WASSEC. This WASC's project is run by Anurag Agarwal and is about the evaluation of web application scanners such as Watchfire's AppScan, SPI's WebInspect etc.
If you are in the field, don't wait to help us :). Here is Anurag's words:
Thank you all for your patience. We have received an overwhelming response from the WASSEC (Web Application Security Scanner Evaluation Criteria) project. To proceed with the project please
1. Please email wasc-wassec-subscribe(AT)webappsec(DOT)org and reply to confirmation email.
2. It is moderated subscription so every contributor has to be approved to send messages to the list.
3. Once you are subscribed to the list, then email wasc-wassec(AT)webappsec(DOT)org to post messages.
All further communication will be done through the mailing list. Please keep checking your junk mail folder in case some messages might go there. We are also in the process of setting up a wiki for the length of the project to post updates, etc. Until then I will be updating my blog with the project details.
Once again, thank you for your participation.
You can checkout the project here: http://webappsec.org/projects/wassec
Wednesday, August 15 2007
By Romain on Wednesday, August 15 2007, 13:18 UTC
Thanks to ExtendeD, I've fixed the comment issues on the website. It's now possible to post again...
Sorry for the problems.
Thursday, July 12 2007
By Romain on Thursday, July 12 2007, 14:12 UTC
I've just received this book, looked over quickly and it seems a must to have!
I really suggest you to buy this book if you are a developer!
Wednesday, June 27 2007
By Romain on Wednesday, June 27 2007, 15:53 UTC
This is a really nice initiative from Christian and Ronald: http://planet-websecurity.org/
This is for now an aggregator for a couple of web security websites (really good ones).
This site will replace 7 rss I already have :)
Thanks guys
Monday, June 18 2007
By Romain on Monday, June 18 2007, 22:35 UTC
After Watchfire acquired by IBM, it seems like HP would be close to SPI-Dynamics!
So, it will be likely IBM vs. HP vs. Cenzic vs. Acunetix... But Cenzic doesn't care, they have the "Fault Injection" patent ^^
By Romain on Monday, June 18 2007, 19:14 UTC
Google has just released the so called "Safe Browser API" which allows everybody to know if a given url is known as a phishing website or malwares infested page. This service is already working with Firefox.
Wednesday, June 6 2007
By Romain on Wednesday, June 6 2007, 11:20 UTC
And they plan to buy it: read the news here
Friday, January 12 2007
By Romain on Friday, January 12 2007, 07:38 UTC
$8000-$12000 for a vulnerability disclosure in Internet explorer 7 or Vista ?
http://labs.idefense.com/vcp/challenge.php
Tuesday, December 19 2006
By Romain on Tuesday, December 19 2006, 17:41 UTC
You can reach the news from the SPI-Dynamics portal here:
http://portal.spidynamics.com/.../IE7-_2D00_-Phishing-vs.-Privacy.aspx
Basically, it shows that Internet Explorer 7 sends information on what you are currently browsing to a microsoft site with SOAP. Isn't it scary ?
Sunday, December 17 2006
By Romain on Sunday, December 17 2006, 17:38 UTC
I had some trouble with this blog, I was thinking it was because this is a dotclear2 beta version, but the tags and the rss links didn't work.
Then, I upgraded to the last beta version... same thing. The problem was actually the URL rewriting in Apache. It should be good now.
Last comments