When extracting hashes from an active directory database for password auditing purposes, it can also possible to extract hashes of a user's password history.

A reader of the ISC diary working for a Fortune 500 company used this password history analysis tool and shares his findings which us, for which I like to thank him.

Report:

Background

I work for a global Fortune 500 company where employees number in the several tens of thousands. Our password policy isn’t monolithic, but at the core it varies some from the nebulous “industry standard”. One of the elements that is different is that we ask users to rotate their passwords more frequently than most. For our most sensitive users, that is every 30 days.

 

Password discussions are numerous out there on the internet and there have been many articles of late suggesting that frequent password rotation just causes users to iterate their passwords on the same core string. Password1 becomes Password2, then Password3 and so on. We had some spirited internal discussions about whether we should relax our policy in accordance with this guidance. Ultimately, we decided that we wanted some data on password security in our environment. People can blog all they want about opinions, but the best security decisions are made with data to support them.

 

So we set out to measure how often our users made use of a shared string when constructing their passwords, and also how ‘secure’ our users passwords actually are. To be clear, I’m using the term “measure” here in the sense that we are trying to reduce uncertainty about a subject, not trying to know exactly the size/shape/scope of a thing. (For more on that, I recommend “How to Measure Anything” by Douglas Hubbard). Also, by measuring now, we get the opportunity to measure the impact any change in our password policy has on our user habits. Double win!

Process

Measuring passwords meant (to us) password cracking. Before going further, I want to be clear that we obtained permission from our Legal department to conduct our password auditing after satisfying them that we were taking due care with user passwords and privacy. I’m not going to go into those details here, but suffice to say that permission is essential for activities like this.

 

One of the things that we wanted to demonstrate was the effectiveness that password cracking could have - and in what timeframe. Accordingly we built a very moderate cracking rig for both financial reasons and also to show what a simple setup could achieve. We bought a motherboard which could handle two modern graphics cards, a cheap Intel Celeron processor, and 2 GeForce GTX 1070 graphics cards. We scraped together some stray RAM, scavenged an old desktop case, and with some ingenuity, crammed the new hardware into the old case. It’s a Frankensteinian monster - nothing sleek or sexy about it. But it’s functional and cost less than $2000 all told.

 

In an enterprise, the best place for a treasure trove of passwords to crack is the ntds.dit file on an Active Directory Domain Controller. That, in combination with the SYSTEM hive gives you access to all the passwords in AD. In our environment, ntds.dit turned out to be 5 GB. That’s...pretty large.

 

We first attempted to follow what seems to be the standard process of using the utility ntdsextract. For example, Didier Stevens walks through this process in his blog series. When we got to the esedbextract stage, we let it run for about 24 hours and had only extracted ~130 MB of data. That was not going to scale well with our environment, so we turned to option 2, secretsdump.py. (Didier also covered this tool). Happily, secretsdump worked much faster and gave us what we wanted within our lifetime.

 

The next important step was to filter out only the historical password. Luckily, our Active Directory environment is configured to retain up to 120 passwords in the password history, so we had plenty of data. The important point is that we never wanted to crack a password currently in use if we could help it.

 

For cracking, we used hashcat, compiled to run on a default install of Ubuntu. We used the Crackstation wordlist and the default rules available in hashcat for a series of hybrid attacks. We tried various iterations with that wordlist as well as some created from observations about our environment in a hybrid attack (wordlists plus rules generated by hashcat). Eventually, we turned to a little brute force cracking to bat cleanup. We brute forced all passwords from 1 through 8 characters in length in our environment. All told, we spent roughly 6 days of cracking time (several weeks of real time juggling other projects and troubleshooting this or that), 5 of which were just the straight brute force attacks.

 

But how to analyze what we set out to do, the use of shared strings across multiple passwords? Didier Stevens to the rescue (again)! While troubleshooting the extraction process, I contacted Didier and explained what we were up to. Like the Santa Claus of security tools, he shared a rough script which he had created to analyze the use of shared strings in passwords. I have been helping him to test and polish it up and he recently released it. Huge thanks to Didier for making awesome tools and being willing to spend time and effort helping us out.

Results

Below are some statistics about what we learned by analyzing our cracked passwords. Please note, these observations are only on the passwords which we cracked, not the 80% which we haven’t (yet). By necessity, we cracked the easy ones first of course, which included the ones with shared strings. So although these observations are certainly interesting, they are necessarily skewed towards users with poor password hygiene. Just something to keep in mind.

 

• 974,111 - the total number of historical password hashes extracted from ntds.dit. A treasure trove as promised!

• 189,000 - total passwords cracked so far, 19.4% of the available pool

• 15,800 - user accounts which had at least one password cracked

• 12 - average number of passwords cracked per user account

• 2 minutes - the time taken for the first pass with a wordlist and 64 rules to crack the first 38,000 passwords

• Just under 5 days - time taken to brute force all passwords up through 8 characters in length

• ~6 days - total time spent by the cracking rig to crack passwords

• 87.8% - passwords cracked via wordlist (in ~24 hours total time)

• 12.2% - passwords cracked via brute force (in ~5 days time). Wordlists are the by far the most efficient use of time!

• 9,489 unique user accounts made use of a shared string between at least 2 passwords - 60% of the total accounts

• 7,568 unique shared strings - 1,921 shared strings were duplicates used by multiple user accounts

• 109,861 passwords that had a shared string - 58.2% of the total passwords cracked

• 61 - the number of successive passwords for a single user account which all shared the same string - several users have not varied their passwords for 5 or more years!

• 27 characters - the longest password cracked; It was a name and digits repeated several times

• 6,400 - the number of passwords which had an element of the company name as a shared string (does not include modified versions of the company name like c0mp@ny)

• 19 characters - the longest shared strings observed. One of them was “Thisisalongpassword” - maybe so, but not good enough to avoid being cracked.

 

Conclusions

With the help of Didier’s script, we certainly learned a lot about the use of shared strings in our user’s passwords. However, we were not able to find the data to conclude anything concrete about the relationship of our password rotation policy and the use of shared strings. That’s ok though, because by running through the cracking process we did show that the difference between a 30 day rotation policy and something longer is not going to make much difference.

 

We clearly demonstrated that a moderate cracking rig run by people who don’t crack passwords as their job can achieve devastating results in very little time. 8 character passwords are not in any way long enough to stave off password cracking as an effective attack against them. A good wordlist and set of rules can effectively crack most of them in minutes or seconds, and brute force can crack all of them within days. No password rotation policy will stay ahead of that.

 

Therefore we should a)make sure that all our passwords are longer than that, b)make sure that it’s really really hard to crack passwords at an enterprise scale (protect your Domain Controllers!) c)adjust our password rotation cycle to ease the burden on our users and d)continue to emphasize the importance of good passwords and educate our users on how to construct them.

 

There are other obvious conclusions that can be made here - two-factor all the things for example. None of these conclusions are ground-breaking or anything that you, as a security professional, did not likely know before reading this article. But this wasn’t really an exercise in demonstrating that “passwords are dead” or the weakness of relying on passwords as a sole means of authentication. It was an attempt to measure user behavior, justify security policy changes with data, and track the impact that adjusting those policies may have on our users/our security posture.

 

In the first two of those goals, we feel that we have succeeded well.

 

I said it before, but it’s worth saying again: thanks to Didier for his contributions to our efforts and the community at large.

Didier Stevens
Microsoft MVP Consumer Security
blog.DidierStevens.com DidierStevensLabs.com