creating an open web application security home
June 1st, 2011 by Chris Lyon
I made a statement in my previous post, SHA-512 w/ per Users Salts about a “significant hit rate” when it comes to dictionary attacking hashes. This significant hit rate is what we are scared of because we feel that not many people really know the ease of dictionary attacking the hashes, even if you have a large salt. It should be known that hashes alone are not meant to secure passwords. Additional steps are required such as increased iterations and salts are necessary to increase the cost of both offline brute force attacks and pre-computed tables. (rainbow tables) As I pointed out in my last post, most applications store the salt with the hash.
So, on my quest to prove a point and to prove how easy it is to dictionary hashes, I designed a system where we could perform dictionary attacks but under the strictest security possible. I didn’t want to use a public cloud nor did I want to know the passwords. My first goal with this project was to get two metrics, how fast could I dictionary 1 million hashes and what would be the hit rate.
My first mission was to get a few systems for testing and since there was plenty of old desktops and Mac Book Pros around the office, I grabbed a few of these and started building my own. The first task was to build a client/server app that got the hash from the master database and then past the hash to the worker. Once that was done, the local server had to have a database for metrics and to keep timing and hit rates. The API that I wrote between the client and server was pretty simple, auth the request, request a hash, and ack the client “got the hash.” The client also needs to be multi-threaded, which is pretty simple at this point. When the worker was completed with a hash, send a true/false for the ability to dictionary the password and how long did it take.
Once this was built, and I am over simplifying the how in this post, I started testing against sample hashes to get an idea of scale. I started off with just three worker machines, all over 1 1/2 years old. I found that I could get an answer on any given hash under 4 seconds. The dictionary that I am using is my own dictionary, something I won’t release out to the public (yet) but I will say, it has 400,000 entries. I do have a more complete dictionary that is over 10 million entries, but it takes some significant time to process this dictionary but has a much higher hit rate.
When I got the system tested and working, I was able to unleash it on 1 millions hashes. The results were pretty surprising in that I was able to completely process 1 million hashes in under 18 hours using just three older machines and get a 20% password hit. I did want to stress that we didn’t record the password, just if we got a match.
Imagine if I had more machines or even used ec2, I could cut that time down significantly. This is the biggest reason we are moving away from sha-512 and moving towards hmac with bcrypt.
On a personal note, I did want to give one last “Thank you” to all the people in the community who I have had a chance to work with over the past 2 years. As many of you know, Friday June 3rd will be my last day at Mozilla as I am moving on to new challenges. The infrastructure security group wasn’t here when I started and I’m proud to say that it is now starting to put its feet down and establish itself as a “security enabler” for Mozilla and the community. The team that I am leaving behind is nothing short of top notch and will continue to be security enablers.
Once again, it has been a great ride and thank you all for your support.
Director of Infrastructure Security (Until June 3rd)