Failing to programatically overwrite a file in an IIS Virtual Directory/Application (file is always locked)

Question

At first I thought I'm facing a very simple task. But now I realized it doesn't work as I imagined, so now I hope you people can help me out, because I'm pretty much stuck at the moment.

My scenario is this (on a Windows 2008 R2 Server):

1. A file gets uploaded 3 times per day to a FTP directory. The filename is always the same, which means the existing file gets overwritten every time.
2. I have programed a simple C# service which is watching the FTP upload directory, I'm using the FileSystemWatcher class for this.
3. The upload of the file takes a few minutes, so once the File Watcher registers a change, I'm periodically trying to open the file, to see if the file is still being uploaded (or locked)
4. Once the file isn't locked anymore, I try to move the file over to my IIS Virtual Directory. I have to delete the old file first, and then move the new file over. This is where my problem starts. The file seems to be always locked by IIS (the w3wp.exe process).

After some research, I found out that I have to kill the process which is locking the file (w3wp.exe in this case). In order to do this, I have created a new application pool and converted the virtual directory into an application. Now my directory is running under a seperate w3wp.exe process, which I supposedly can safely kill and move the new file over there.

Now I just need to find the proper w3wp.exe process (there are 3 w3wp.exe processes running in total, each running under a seperate application pool) which has the lock on my target file. But this seems to be an almost impossible task in C#. I found many questions here on SO regarding "Finding process which locked a specific file", but none of the answers helped me. Process Explorer for example is exactly telling me which process is locking my file.

The next thing I don't understand is, that I can delete the target file through Windows Explorer without any problem. Just my C# application gets the "File is being used by another process" error. I wonder what's the difference here...

Here are the most notable questions on SO regarding locked files and C#:

Win32: How to get the process/thread that owns a mutex?

^^ The example code here does actually work, but this outputs the open handle IDs for every active process. I just can't figure out how to search for a specific filename, or at least resolve the handle ID to a filename. This WinAPI stuff is way above my head.

Using C#, how does one figure out what process locked a file?

^^ The example code here is exactly what I need, but unfortunately I can't get it to work. It is always throwing an "AccessViolationException" which I can't figure out, since the sample code is making extensive use of WinAPI calls.

Simple task, impossible to do? I appreciate any help.

EDIT Here are some relevant parts of my server code:

Helper function to detect if a file is locked:

    private bool FileReadable(string file, int timeOutSeconds)
    {
        DateTime timeOut = DateTime.Now.AddSeconds(timeOutSeconds);

        while (DateTime.Now < timeOut)
        {
            try
            {
                if (File.Exists(file))
                {
                    using (FileStream fs = File.Open(file, FileMode.Open, FileAccess.Read, FileShare.None))
                    {
                        return true;
                    }
                }
                return false;
            }
            catch (Exception)
            {
                Thread.Sleep(500);
            }
        }

        m_log.LogLogic(0, "FileReadable", "Timeout after [{0}] seconds trying to open the file {1}", timeOutSeconds, file);
        return false;
    }

And this is the code in my FileSystemWatcher event, which is monitoring the FTP upload directory. filepath is the newly uploaded file, targetfilepath is the target file in my IIS directory.

        // here I'm waiting for the newly uploaded file to be ready
        if (FileReadable(filepath, FWConfig.TimeOut))
        {
            // move uploaded file to IIS virtual directory
            string targetfilepath = Path.Combine(FWConfig.TargetPath, FWConfig.TargetFileName);

            if(File.Exists(targetfilepath))
            {
                m_log.LogLogic(4, "ProcessFile", "Trying to delete old file first: [{0}]", targetfilepath);
                // targetfilepath is the full path to my file in my IIS directory
                // always fails because file is always locked my w3wp.exe :-(
                if(FileReadable(targetfilepath, FWConfig.TimeOut))
                    File.Delete(targetfilepath);
            }

            File.Move(filepath, targetfilepath);
        }

EDIT2: Killing the w3wp.exe process while clients are downloading the file would be no problem for us. I'm just having a hard time finding the right w3wp.exe process which is locking the file.

Also, my client application, which is downloading the file on the clients, is checking the HTTP HEAD for the Last-Modified date. The client is checking the date every 10 minutes. So it is possible that the file is being locked by IIS because there are clients continously checking the HTTP HEAD for the file. Nonetheless, I don't understand why I can manually delete/rename/move the file through windows explorer without any problems. Why does this work, and why does my application get a "Locked by another process" exception?

Answer 1

One problem I've run into is that a file exists while it is still being written, which means it would be locked as well. If your FileReadable() function were called at this time, it would return false.

My solution was to, in the proc which writes the file, write the file to, say, OUTPUT1.TXT, and then after it is fully written and the FileStream closed, rename it to OUTPUT2.TXT. This way, the existence of OUTPUT2.TXT indicates that the file is written and (hopefully) unlocked. Simply check for OUTPUT2.TXT in your FileReadable() loop.

Answer 2

Everybody say...

"Do it a better way"

Nobody say how!!!

Here's how. Because you mentioned 'My Client Application,' there is a key opportunity here that you would not have if you didn't have control over the apps reading the file.

Just use new filenames each time.

You have control of the program reading and writing the files. Put an incrementing # in the filesnames, have the client pick the biggest # (Actually the latest date, then your numbers can wrap around). Have the writer program clean up old files if it can; if not, they won't hurt anything. IIS will eventually let go of them. If not, just open up explorer every week and do it yourself!

Other keys that make this work are the low frequency of updates (files won't build up too bad), and the fact that the FTP+webserver are on the same drive (Otherwise the MOVE is not atomic and clients could get a half-copied file. Solution if FTP drive is different would be to copy to a temp drive on the webserver then move).

but what if you can't change the client or it has to read just one name?

Front-end it with a script. Have the client hit an ASPX that sets the right HTTP headers and has the 'pick the right file' logic, and spits out the file contents. This is a very popular trick pages use to write images stored on a database out to the browser, while the img tag appears to read from a file. (google along that lines for sample code).

sounds like a hack, it's not. Modern lockless memory cache systems do a similar thing. It is impossible for a lock or corruption to occur; until the 'write' is complete, readers see the old version.

plus, it's simple, everybody from a script kiddie to a punchcard vetern will know exactly what you're up to. Go low-tech!

Answer 3

You're troubleshooting a symptom of the problem not a fix for the root cause. If you want to go down that path here is the code to kill processes http://www.codeproject.com/Articles/20284/My-TaskManager - but the better idea would be to do it properly and work out whats wrong. I suggest in the Catch Exception of FileReadable:

catch (Exception ex) {
if (ex is IOException && IsFileLocked(ex)) {
//Confirm the code see's it as a FileLocked issue, not some other exception
//its not safe to unlock files used by other processes, because the other process is likely reading/writing it. 
}
}

private static bool IsFileLocked(Exception exception)
{
    int errorCode = Marshal.GetHRForException(exception) & ((1 << 16) - 1);
    return errorCode == 32 || errorCode == 33;
}

1. Turn off any Anti-Virus software and re-test
2. Increase the polling timeout duration to see if its just a timing thing
3. Check the FTP logfile and see the status for the disconnected client and compare the status code with the ones here.

Answer 4

I don't see in your sample code where you are closing your file stream. Keeping the file stream open will keep a lock on the file. It would be a good idea to close the stream. You probably don't want to be killing your w3wp.exe process, as others here have mentioned.

Answer 5

restarting IIS can unlock the file taken by w3wp.exe.

cmd (run as administrator) -> iisreset /stop -> update/delete file in windows explorer -> iisreset /start