Implement a Data Structure to Compare NTLM Hashfiles

Question

Currently I'm in the process of simplifying a process for extracting Windows password hashes in security audits. Personally I want to make the process easier to generate a list of recovered users and their passwords when I do an audit. I think it would also be useful for other people who are trying to compare and generate large amounts of data.

So here's the gist:

When I extract all of the data from the Windows system files, I simplify them down to the format user:hash, where the hash is an NTLM hash such as "a87f3a357d73085c45f9416be5787e86."

I then will use oclHashcat and attempt to crack the hashes, whether it be dictionary or brute-force, it doesn't matter. I generate an output of all of the recovered hashes, however Hashcat generates them in the format hash:password.

Now here's my problem and what I would like some input on - I want to produce the output as user:password given the two input files. Considering that I can have hundreds of hashes yet only a few recovered passwords, there is no use of trying to order the lists.

I am unsure which data structure might benefit me the most. Arrays were too inefficient for large tables. I've looked into serialization and I've been exploring the use of Hash Maps and Hash Tables. Given the size of the hash, I haven't had any luck implementing either of these methods, or I'm doing so incorrectly.

Currently I'm running the program like so:

program [user:hash file] [hash:password file] -o [user:password output]

And I'm effectively trying to run the program like so (briefly):

Load Files

// user:hash file
For each line, split by ':' delimiter
 before delimiter = table1.user
 after delimiter = table1.hash

// hash:password file
For each line, split by ':' delimiter
 before delimiter = table2.hash
 after delimiter = table2.password

// generate user:password file
Check each entry of table1 vs table2
 if table1.hash = table2.hash
  table1.user =  output.user
  table2.password = output.password
  print to output "output.user:output.password"

I am only trying to figure out an efficient method for tracing through each line and extracting the necessary data into a data structure that I can easily trace through.

If I need to clarify anything, please let me know. Any help is appreciated!

Answer 1

I decided to go with a shell script, and I used an associated array to match the required data that I needed.

Note: Most of this program deals with the way my hash files are formatted. Look at the comments in the script.

#!/bin/bash
# Signus
# User-Password Hash Comparison Tool v1.0
# Simple utility that allows for the comparison between a file with a 'user:hash' format to a separate file with a 'hash:password' format. The comparison matches the hashes and returns an output file in the format 'user:password'

# Example of 'user:hash' -> george:c21cfaebe1d69ac9e2e4e1e2dc383bac
# Example of 'hash:password' -> c21cfaebe1d69ac9e2e4e1e2dc383bac:password
# 'user:hash' obtained from creddump suite: http://code.google.com/p/creddump/
# Note: Used custom 'dshashes.py' file: http://ptscripts.googlecode.com/svn/trunk/dshashes.py
# 'hash:password' obtained from ocl-Hashcat output

usage="Usage: $0 [-i user:hash input] [-t hash:password input] [-o output]"

declare -a a1
declare -a a2
declare -A o

index1=0
index2=0
countA1=0
countA2=0
matchCount=0

if [ -z "$*" ]; then
  echo $usage
    exit 1
fi

if [ $# -ne 6 ]; then
    echo "Error: Invalid number of arguments."
    echo $usage
    exit 1
fi

echo -e
echo "---Checking Files---"
while getopts ":i:t:o:" option; do
    case $option in
        i) inputFile1="$OPTARG"
        if [ ! -f $inputFile1 ]; then
            echo "Unable to find or open file: $inputFile1"
            exit 1
        fi
        echo "Reading...$inputFile1"
        ;;
        t) inputFile2="$OPTARG"
        if [ ! -f $inputFile2 ]; then
            echo "Unable to find or open file: $inputFile2"
            exit 1
        fi
        echo "Reading...$inputFile2"
        ;;
        o) outputFile="$OPTARG"
        echo "Writing...$outputFile"
        ;;
        [?]) echo $usage >&2
            exit 1
        ;;
    esac
done
shift $(($OPTIND-1))


#First read the files and cut each line into an array
echo -e
echo "---Reading Files---"
while read LINE
do
    a1[$index1]="$LINE"
    index1=$(($index1+1))
    countA1=$(($countA1+1))
done < $inputFile1
echo "Read $countA1 lines in $inputFile1"

while read LINE
do
    a2[$index2]="$LINE"
    index2=$(($index2+1))
    countA2=$(($countA2+1))
done < $inputFile2
echo "Read $countA2 lines in $inputFile2"


#Then cut each item out of the array and store it into separate variables
echo -e
echo "Searching for Matches..."
for (( j=0; j<${#a2[@]}; j++ ))
do
    hash2=$(echo ${a2[$j]} | cut -d: -f1)
    pword=$(echo ${a2[$j]} | cut -d: -f2)

    for (( i=0; i<${#a1[@]}; i++ ))
    do
        us=$(echo ${a1[$i]} | cut -d: -f1)
        hash1=$(echo ${a1[$i]} | cut -d: -f2)

        if [ "$hash2" = "$hash1" ]; then
            matchCount=$(($matchCount+1))
            o["$us"]=$pword
            echo -e "Match Found[$matchCount]: \t Username:$us  \t  Password:$pword" 
            break
        fi

    done
done

echo -e "Matches Found: $matchCount\n" >> $outputFile
for k in ${!o[@]}
do
    echo -e "Username: $k  \t  Password: ${o[$k]}" >> $outputFile
done
echo -e "\nWrote $matchCount lines to $outputFile"

unset o

Answer 2

I would do this for my data structure

std::map<std::string,std::string> hash_user_map;

No go through all the users and hashes from table 1

For each user in table1
hash_user_map[table1.hash] = table1.user;

No go through all the cracked passwords with hashes from table2

std::string user = hash_user_map[table2.hash];
std::cout << user << ":" << table2.password << "\n;