PowerShell: Searching for a part of a string in a logfile

Question 1

Are you sure it's actually working on Windows 7? They way you have it written, each iteration of the foreach loop overwrites the matchfile with the results from the last file in $files. You could add the -Append switch after Out-File, but you don't even need the foreach loop. Also, what's the purpose of $match? The $match = looks superfluous to me. Try this:

Get-Content $files | Select-String $p | Out-File $matchfile

I suspect that the reason you're not getting any results on the 2K8 server is that the last file in $files there has no matches for $p. On the Windows 7 machine, you're getting results because the last file has matches for $p, but you're not getting results from any of the other files.

Question 2

I second @Adi, you are overwriting the file in each iteration. You can solve this by adding the Append switch to Out-File. This should also work and is simpler to use:

$files | Select-String $p | Out-File $matchfiles

Question 3

I would expect that $match will always be null. According to the help, Out-File never sends anything down the pipeline. But I'd be surprised if that was related to your problem.

In the spirit of "Any help would be greatly appreciated"...

I tried a simple experiment on both Win7 and WS08R2 and couldn't repro your results (it worked as expected). I also tried creating the file being searched with different encodings: Unicode, UTF8, and ASCII. All worked. I even told Select-String to use the wrong encoding:

get-content mytestfile.ascii | select-string error -Encoding Unicode

and it still worked. I used a hex viewer to verify that mytestfile.ascii, mytestfile.unicode, and mytestfile.utf8, were all in fact encoded differently. I also tried different line endings (Linux style vs Windows style) and that worked.

My guess is that you have something unique about the files, the system, etc. that will have to be discovered by some probing. With that in mind...

I suggest you look inside one of your logfiles with a hex viewer (try it from both systems) to see if something is different between your Win7 system and your WS08R2 system. At the beginning of a UTF8 file you will see EF BB BF and then one byte for each character of your log. At the beginning of a Unicode file you will see FF FE and then two bytes for every character. (If the text is English then every other character will be 0). And for an ASCII file there isn't a special value at the start and there is one byte per letter. Linux line endings are a single 0A (linefeed) and Windows line endings are 0D 0A (carriage return, linefeed). Besides special characters at the beginning as mentioned above, and line endings you'd be looking for the text to be exactly the same set of characters as if you viewed it in notepad. (BTW, viewing it in notepad will only look "right" if line endings are Windows style). If you see any extra characters, then that might be the source of the problem

If that doesn't show anything unexpected, then I suggest a series of experiments from Powershell command line. The following procedure has a lot of steps listed, but it tries to cover a lot of contingencies. You won't be performing every step. (However with a couple of modifications at the beginning - to account for where "error" appears in the log file you use to test with - you could copy and paste this whole script and then interpret the whole set of results at one time)

# 0. Get some text to experiment with
#    If "error" appears on line 200 and line 493 (where first line in file is Line 1), then:
$testTxt= cat mylogfile | select-object -skip 195 -first 300
# or better still:
$testTxt= cat mylogfile | select-object -skip 195 -first 10
$testTxt2= cat mylogfile | select-object -skip 488 -first 10
$testTxt += $testTxt

# 1. Figure out where in testTxt "error" appears
$testTxt[3..5] # if my math is right "error" will appear as middle line

# 2. If #1 doesn't work on both systems, then on failing system(s):
$testTxt[4] | select-string error

# 2.5 If #2 fails then you need to look at every character
#  of $testTxt
[char[]]$testTxt 
# or even look at every byte value in hex:
[byte[]][char[]]$textTxt | % { $_.tostring("X")}

# 3. If #2 worked, then:
$testTxt | select-string error

# 4. If #3 failed, then:
$testTxt[3..5] | select-string error

# 5. If #4 failed, then:
$testTxt[3..4] | select-string error
$testTxt[4..5] | select-string error

# 6. If either of #5 failed, then carefully examine those lines of text:
"line 3"
[char[]]$testTxt[3] 
"line 4"
[char[]]$testTxt[4] 
"line 5"
[char[]]$testTxt[5] 

# 7. If #3 worked then:
$testTxt2 = cat myLogFile
$testTxt2 | select-string error

# 8. If #7 worked, then you have a solution
# Replace:
get-content $file | Select-String $p | Out-File $matchfile
# With (on Powershell v3 this may perform worse than the above, assuming above worked)
$logText = get-content $file
$logText | Select-String $p | Out-File $matchfile

# 9. If #8 failed, then the situation is somewhat awkward.
#    On one hand you have twenty lines working, but on the other the file as a whole
#    doesn't work. You could go back to step #0 and start with a larger amount of text,
#    say 40 lines, and see if it works. If it does you could try 80, 160, etc. until it
#    fails. But this case seems pretty unlikely.

# 10. In general when a step fails you want to try something "smaller". And when steps
#     succeed you want to try something "bigger", where "bigger" means more like the 
#     solution you want.