How to detect changes to files recursively?

Question

I'm working on a multi-threaded component to load and manage a music library, and I have a property defining multiple root directories to include. One thread searches those directories for media files, adds/removes as necessary, and another thread goes through those files and fills in the ID3v2 tag information. I already have a mechanism to detect added/removed files, but I don't know how to detect changes.

How can I detect when changes have been made to any of these files from other outside applications? I'd like an instantaneous response and not have to wait for a thread to get to that file. Is there a way I can receive alerts when any files have been changed in any of these folders recursively?

Answer 1

The function that you need to use is ReadDirectoryChangesW. This is not the easiest function in the world to use and it is worth pointing out that it is not 100% reliable. It will sometimes fail to notify you of modifications. In my experience that is more likely to happen for shares.

This API can be used in synchronous or asynchronous modes. As is always the case, the synchronous version is much easier to code against. But of course it blocks the calling thread. So the way out of that is to put the calls to ReadDirectoryChangesW in different threads. If you have a very large number of directories to watch, then one watching thread per directory is going to be an unworkable burden. If that is so then you would need to grapple with asynchronous usage.

You bWatchSubtree parameter allows you to monitor an entire tree of directories which I think is what you want to do.

For more details I refer you to this article: Understanding ReadDirectoryChangesW.

Answer 2

Try this:

uses
  ShlObj, ActiveX;

const
  FILE_LIST_DIRECTORY   = $0001;
  cDir = 'E:\...'; // The directory to monitor

Type
  PFileNotifyInformation = ^TFileNotifyInformation;
  TFileNotifyInformation = Record
    NextEntryOffset: DWORD;
    Action: DWORD;
    FileNameLength: DWORD;
    FileName: Array[0..0] of WideChar;
  End;

type
  TWaitThread = class(TThread)
  private
    FForm: TMainForm;
    procedure HandleEvent;
  protected
    procedure Execute; override;
  public
    constructor Create(Form: TMainForm);
    Procedure SendFtp(F: String; AddIfError: Boolean);
  end;


procedure TWaitThread.HandleEvent;
  Var
  FileOpNotification: PFileNotifyInformation;
  Offset: Longint;
  F: String;
  AList: TStringList;
  I: Integer;
begin

  AList := TStringList.Create;

  Try

    With FForm Do
    Begin

      Pointer(FileOpNotification) := @FNotificationBuffer[0];

      Repeat
        Offset := FileOpNotification^.NextEntryOffset;
        //lbEvents.Items.Add(Format(SAction[FileOpNotification^.Action], [WideCharToString(@(FileOpNotification^.FileName))]));

        F := cDir + WideCharToString(@(FileOpNotification^.FileName));

        if AList.IndexOf(F) < 0 Then
        AList.Add(F);

        PChar(FileOpNotification) := PChar(FileOpNotification)+Offset;

      Until Offset=0;

      For I := 0 To AList.Count -1 Do
      // do whatever you need

    End;

  Finally
    AList.Free;
  End;

end;


constructor TWaitThread.Create(Form: TMainForm);
begin
  inherited Create(True);
  FForm := Form;
  FreeOnTerminate := False;
end;

procedure TWaitThread.Execute;
  Var
  NumBytes: DWORD;
  CompletionKey: DWORD;
begin

  While Not Terminated Do
  Begin

    GetQueuedCompletionStatus( FForm.FCompletionPort, numBytes, CompletionKey, FForm.FPOverlapped, INFINITE);

    if CompletionKey <> 0 Then
    Begin
      Synchronize(HandleEvent);

      With FForm do
      begin
        FBytesWritten := 0;
        ZeroMemory(@FNotificationBuffer, SizeOf(FNotificationBuffer));
        ReadDirectoryChanges(FDirectoryHandle, @FNotificationBuffer, SizeOf(FNotificationBuffer), False, FNotifyFilter, @FBytesWritten, @FOverlapped, nil);
      End;

    End
    Else
    Terminate;

  End;

end;


{MainForm}

  private

    FDirectoryHandle: THandle;
    FNotificationBuffer: array[0..4096] of Byte;
    FWatchThread: TThread;
    FNotifyFilter: DWORD;
    FOverlapped: TOverlapped;
    FPOverlapped: POverlapped;
    FBytesWritten: DWORD;
    FCompletionPort: THandle;


procedure TMainForm.FormCreate(Sender: TObject);
begin

  FCompletionPort := 0;
  FDirectoryHandle := 0;
  FPOverlapped := @FOverlapped;
  ZeroMemory(@FOverlapped, SizeOf(FOverlapped));

  Start;

end;

procedure TMainForm.Start;
begin

  FNotifyFilter := 0;

  FNotifyFilter := FNotifyFilter or FILE_NOTIFY_CHANGE_FILE_NAME;

  FDirectoryHandle := CreateFile(cDir,
                      FILE_LIST_DIRECTORY,
                      FILE_SHARE_READ or FILE_SHARE_WRITE or FILE_SHARE_DELETE,
                      Nil,
                      OPEN_EXISTING,
                      FILE_FLAG_BACKUP_SEMANTICS or FILE_FLAG_OVERLAPPED,
                      0);

  if FDirectoryHandle = INVALID_HANDLE_VALUE Then
  Begin
    Beep;
    FDirectoryHandle := 0;
    ShowMessage(SysErrorMessage(GetLastError));
    Exit;
  End;

  FCompletionPort := CreateIoCompletionPort(FDirectoryHandle, 0, Longint(pointer(self)), 0);
  ZeroMemory(@FNotificationBuffer, SizeOf(FNotificationBuffer));
  FBytesWritten := 0;

  if Not ReadDirectoryChanges(FDirectoryHandle, @FNotificationBuffer, SizeOf(FNotificationBuffer), False, FNotifyFilter, @FBytesWritten, @FOverlapped, Nil) Then
  Begin
    CloseHandle(FDirectoryHandle);
    FDirectoryHandle := 0;
    CloseHandle(FCompletionPort);
    FCompletionPort := 0;
    ShowMessage(SysErrorMessage(GetLastError));
    Exit;
  End;

  FWatchThread := TWaitThread.Create(self);
  TWaitThread(FWatchThread).Resume;

end;

procedure TMainForm.Stop;
begin

  if FCompletionPort = 0 Then
  Exit;

  PostQueuedCompletionStatus(FCompletionPort, 0, 0, Nil);
  FWatchThread.WaitFor;
  FWatchThread.Free;
  CloseHandle(FDirectoryHandle);
  FDirectoryHandle := 0;
  CloseHandle(FCompletionPort);
  FCompletionPort := 0;

end;

procedure TMainForm.FormDestroy(Sender: TObject);
begin
  Stop;
end;