How do I select multiple columns and expand properties using powershell select-string -Allmatch?

Question

I'm searching a collection of text files (*.sql) for the occurrence of 8 digit numbers that begin with a 9. There could be multiple instances of these numbers in the file as well as multiple instances on each line in the file. I only want the output to display the unique occurrence of each 8 digit number in each file. This is what I have so far:

Select-String "9[0-9]{8}" "*.sql" -AllMatches | Select-Object FileName, @{N="Value";E={ $_.matches |  %{$_.groups[0].value}}} | Select-Object  -unique FileName,Value  

And my output look like this:

FileName                       Value
--------                       -----

File1.sql                      907520714
File1.sql                      {907500507, 907520700, 907520701, 907520703...} 
File1.sql                      {907520725, 907520727, 907520728, 907520729} 
File1.sql                      990140600
File2.sql                      990319161
File2.sql                      {990603919, 990603925, 990603926} 
File2.sql                      {991100103, 991100103}
File2.sql                      {990700023, 990700504, 990700521, 990740520...} 
File3.sql                      907500044

etc....

What I would like to do is expand the arrays so that when I pipe it through select -unique I actually will only get the unique 8 digit numbers contained in each file.

This is what I would like:

FileName                       Value
--------                       -----

File1.sql                      907520714
File1.sql                      907500507
File1.sql                      907520700
File1.sql                      907520701
File1.sql                      907520703 
File1.sql                      907520725
File1.sql                      907520727
File1.sql                      907520728
File1.sql                      907520729 
File1.sql                      990140600
File2.sql                      990319161
File2.sql                      990603919
File2.sql                      990603925
File2.sql                      990603926 
File2.sql                      991100103
File2.sql                      990700023
File2.sql                      990700504
File2.sql                      990700521
File2.sql                      990740520 
File3.sql                      907500044

etc...

How can I do this? And can my current powershell command be improved?

Thanks!

Answer 1

I reformatted the command a bit, I think the terse nature of PowerShell shorthand makes it harder to see the problem.

Select-String "9[0-9]{8}" "*.sql" -AllMatches | `
    Select-Object FileName, @{N="Value";E={ $_.matches | %{$_.groups[0].value}}} | `
    Select-Object  -unique FileName,Value 

Your original command yields one output line for each MatchInfo returned by Select-String. A MatchInfo represents a matching line of the text file. I believe you are getting an array of values when a single line in the file contains more than one matching value.

I revised it to clarify that we want one output object for each RegexMatch in each MatchInfo, which is to say one output object for each matching value.

Select-String "9[0-9]{8}" "*.sql" -AllMatches | `
    Foreach-Object { 
        # $_ is MatchInfo for each matching line in file
        $fileName = $_.FileName 
        $_.Matches | Foreach-Object { 
            # $_ is RegexMatch for each match in line
            $_ | Select-Object -Property @{N="FileName";E={$fileName} },Value
        } 
    } | Select-Object  -unique FileName,Value 

Answer 2

Ok, mostly running off what you have, kind of. I'm doing a ForEach loop against all of the matches found by the Select-String, and grouping them by file name. Then for each file I loop through the groups expanding Matches, and then take only unique values for the matched text. For each value I output an object containing the name of the file, and the value that was matched.

ForEach($File in Select-String "9[0-9]{8}" "*.sql" -AllMatches | Group FileName){
    $File.group|select -expand matches|Select Value -Unique|%{
        new-object PSObject -property @{
            FileName=$File.Name
            Match=$_.value
        }
    }
}