Detecting FTP timeout in Windows batch script

Question

I have a SQL Server Agent job scheduled to run daily. The first of two steps calls an SSIS package, which writes the results of a query to a file. The second step executes a batch file which renames the file to include the date, uploads the file to a partner's FTP site, moves the file to an archive directory and cleans up. This normally works fine.

Occasionally, however, no notification is sent that the job completed successfully. When I log into the server, I see that ftp.exe is still running, but the file has not been uploaded nor moved to the archive. When I terminate the FTP process, the job continues, moving the file and cleaning up. A look at the job history reveals two different scenarios in these failed cases. First, the connection is made, but the last FTP output seen is 150 Opening data channel for file transfer.--there is no 226 Transfer OK as there is for a successful job run. The second scenario is that the connection is never established, so the script is then off and trying to log in when there is no connection open.

Here is the script with host, user, and pass munged:

@echo off
setlocal

D:
cd D:\Backups

set file_name=ssis_output.txt
set new_file_name=SDCL.%date:~-4,4%-%date:~-10,2%-%date:~-7,2%-00-00-00.txt

set user=XXXXXXXXXXXXX
set pass=XXXXXXXXXXXXX
set host=XXXXXXXXXXXXX

ren "%file_name%" "%new_file_name%"

echo user %user%> ftpcmd.dat
echo %pass%>> ftpcmd.dat
echo bin>> ftpcmd.dat
echo put %new_file_name%>> ftpcmd.dat
echo close>> ftpcmd.dat
echo bye>> ftpcmd.dat
ftp -n -s:ftpcmd.dat %host%
del ftpcmd.dat
move %new_file_name% data_files

What I would like to do is find a way to include some error detection in the batch file, so that if the FTP connection is either not established or times out before the transfer completes, the script recognizes that and fails, allowing the job to fail and send the appropriate notification. Even better, a connection failure would lead to a few additional attempts after appropriate pauses before finally surrendering and reporting a failure.

EDIT: I'm not opposed to installing a different freeware FTP client, such as WinSCP, if that would aid in arriving at a solution.

Answer 1

I spent the weekend casting about for some way to get the Windows FTP client to exit on a connection failure, but was unable to find anything. I'm pretty sure that it is not possible. In lieu of that, I installed WinSCP, which has a console mode and is decently scriptable. It has a reconnecttime option that allows you to specify the number of seconds the process will wait before attempting to reconnect. In testing, the job was completed successfully even when I disconnected the FTP session from the server side several times. A permanent disconnection caused the program to exit with an exit code of 1, which is sufficient for the SQL Server agent to detect that the job failed and notify us accordingly.

WinSCP doesn't seem to be consistent, though, in its handling of disconnections. In some trials, a second disconnection of a session caused the program to exit with 1, while in others it reconnected after many disconnections. I landed on these settings as the ones maximizing the probability of reconnections and successful job completion:

option batch on
option reconnecttime 30
option confirm off

Answer 2

Might be a little late for OP. But to log FTP you could have it write confirmation to a log file every time it uploads a file. Then after the script completes, have it check the log file to see if the transfer completed successfully. If it didn't, have it resume from the last known transferred file. Could be a pretty messy loop for batch scripting though.