search through 10,000s of log files for a specific string then out put each line that string is in while also creating a copy of that log file

StackOverflow https://stackoverflow.com/questions/21315643

Question

There's a few questions already like this but none were specific enough for my purposes.

I need to search through 10,000s of log files for a specific string then out put each line that string is in while also creating a copy of that log file.

I have it almost working in a BATCH file .. I think I hit my wall and need to start using powershell which I haven't used much before.

:update

Thanks to Trondh , I was able to use his script as a perfect base and put in the features I needed. Hopefully this helps someone else :)

#Folder to search
$folder = read-host "Please specify the location of the search "

#Search for: 
$SearchTerm = read-host "type in the word you want to find Eg. Error or JobID "

#Files to include in search
$FileFilter = read-host "Enter Date Part of file filter Eg. 20140123 or 201401 "

#File to store log file copies in
$destinationfolder = "Backup-$SearchTerm"

#File to store results in
$newfile = "Search-Log-$SearchTerm.txt"

#Get the files according to filter. Recurse, but exclude directories
$files = Get-ChildItem -Path $folder -Include @("*$filefilter*.*") -recurse | where {$_.PSIsContainer -eq $false}
foreach ($file in $files)
    {
        $result = $file | Select-String $SearchTerm

        $result | add-content $newfile

        New-Item -ItemType Directory -Force -Path $destinationfolder  

        #If we get a hit, copy the file
        if ($result)
            {
                Write-host "Found match in file $($file.Name) ($($file.Directory))"
                #Add result to file
                $file | Copy-Item -Destination $destinationfolder 


                #Also output it
                $result 

            }

    }


   Write-Host "Search Completed!"

$x = $host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")
Was it helpful?

Solution

Here's how I would do it:

#Folder to search
$folder = "D:\trond.hindenes\Desktop\test"
#File to store log file copies in
$destinationfolder = "D:\trond.hindenes\Desktop\test2"
#Search for:
$SearchTerm = "BAT"
#Files to include in search
$FileFilter = "file*"

#Get the files according to filter. Recurse, but exclude directories
$files = Get-ChildItem -Path $folder -Include $filefilter -recurse | where {$_.PSIsContainer -eq $false}
foreach ($file in $files)
    {
        $result = $file | Select-String $SearchTerm

        #If we get a hit, copy the file
        if ($result)
            {
                Write-host "Found match in file $($file.Name) ($($file.Directory))"
                #Add result to file
                $file | Copy-Item -Destination $destinationfolder

                #Also output it
                $result

            }

    }

OTHER TIPS

....
....
rem search for files that contain data
for /f "tokens=*" %%f in ('findstr /s /i /m /c:"%word%" "%drive%\*%data%*.log"') do (

    rem copy the file selected by findstr to target folder
    copy "%%~ff" "%targetFolder%"

    rem and echo the lines with the data to the results file
    findstr /n /i /c:"%word%" "%%~ff" >> "Search-Logs-Results-%word%.TXT"
)

findstr /m just tests for the presence of the string in the file, and leave the file on first match, writing the filename to stdout. The list of files is processed with the for command. For each one, the file is copied and then the lines with the required word are sent to the report file.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top