Powershell from SQL Server Agent Job doesn't recognize ConvertTo-CSV

Question

UPDATE: Modified the script to work within the bounds of PS1 as required by SQLPS.

Changed:

 IF($property.Value -match $regex){ 
                   $currentBadLine = (ConvertTo-Csv $_  -NoTypeInformation -Delimiter $delimiter);

                   $badLines += $currentBadLine[1,2]
            };

To:

IF($property.Value -match $regex){ 
              $badLines += $_ | Select-Object | ft -autoSize;
       };

Prints a new header for each bad line, but it's not the end of the world and not worth the effort to prevent.

---

I have a Powershell script that pre-processes CSV files before they have a chance to screw up my data imports.

On two servers in a row now I have confirmed that the PS Major Version at least 2, and that the following code snippet runs fine in Powershell ISE. The purpose of the code is to read in each line of the CSV, and then loop through the columns looking for the regex pattern in the $regex variable. When it finds one I want it to keep track of the error before fixing it so I can write an error log later before outputting a cleaned up file ready for import.

%{
    Foreach($Property in $_.PSObject.Properties){

            IF($property.Value -match $regex){ 
                   $currentBadLine = (ConvertTo-Csv $_  -NoTypeInformation -Delimiter $delimiter);

                   $badLines += $currentBadLine[1,2]
            };

        $property.Value = $property.Value -replace $regex;
      } 

    $_

 }

But once I put that code into an agent job the agent complains:

'The term 'ConvertTo-Csv' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. '

The question then is this: Is the Agents Powershell subsystem using a different version of Powershell than the rest of the system? If so, how do I find out which version the Subsystem is using and if possible upgrade it so I can fix this.

The server is running:

Windows Server 2008 R2 Enterprise

PS Major version 2, Minor 0 Build -1 revision -1

SQL Server 2008 R2 SP1 10.5.2500.0 64 bit

Thanks!

Answer 1

Yes, proper PowerShell support isn't really implemented until SQL Server 2012 (an even that is a bit flakey as to what cmdlets it supports)

In 2008 and R2 the agent's powershell implementation is actually a minishell, created by the now (thankfully) deprecated make-shell.exe utility, which only allows v1 cmdlets to run, and disallows the use of Add-PSSnapin so you can't add anymore cmdlets.

To get proper powershell support, you either need to shell out and call an external script, or run the job as a windows scheduled task rather than an agent job.

The following article explains a lot about why powershell support in 2008 R2 doesn't work like you think it should:

The Truth about SQLPS and PowerShell V2

Answer 2

One work-around: Export-CSV to a file, then Get-Content from the file.

$rows = ,(New-Object PSObject -prop @{a=7; b='stuff';});
$rows +=,(New-Object PSObject -prop @{a=77; b='more';});

#To run from SQL Job, change from this:
$csvrows = $rows | ConvertTo-CSV -NoType | % {$_.Replace('"','')};
write-output $csvrows;

#to this:
$rows | Export-CSV -NoType "C:\Temp\T.csv"
$csvrows = (Get-Content "C:\Temp\T.csv") | % {$_.Replace('"','')};
write-output $csvrows;