How to manage concurrent Input/Output access to a XML file from multiple instances of an EXE, using Delphi.

Question

I have a command line tool, written in Delphi, which job is to insert a node in a XML file and then immediately exit. I need to make it possible several instances of the tool to be executed simultaneously and insert nodes to one and the same XML.

To achieve this purpose I have introduced a simple file "mutex" - the tool creates one temp file before writing to the XML and then deletes the temp file after finished witing. So if another instance is executed, it checks for the presence of this temp file and waits until it is deleted. Then it creates again the temp file, writes to the XML and deletes the temp file.

The problem is that this works fine only when 2-3 instances try to write to the XML file simultaneously. When there are more instances - some of them just wait forever and never append the node into the XML.

Is there a better way to make it work with large number of instances running and writing to the XML at the same time?

Answer 1

A named semaphore or mutex can do this for you on a single machine. Use e.g. TMutex from SyncObjs, and use one of the constructors which takes a name argument. If you use the same name in all the applications, they will be synchronizing over the same kernel mutex. Use TMutex.Acquire to access, and TMutex.Release when you're done, protected in a try/finally block.

Use the TMutex.Create overload which has an InitialOwner argument, but specify False for this (unless you want to acquire the mutex straight away, of course). This overload calls CreateMutex behind the scenes. Look in the source to SyncObjs and the documentation for CreateMutex for extra details.

Answer 2

1 - Set up a file which records pending changes (it will work like a queue)

2 - Write a simple app to watch that file, and apply the changes to the XML file

3 - Modify the current command-line tool to append their change requests to the "Pending changes" file

Now only one app has to touch the final XML file.

Answer 3

TXMLDocument already prevents multiple instances from writing to the same file simultaneously. So I'm guessing that what your question really means is, "How can I open an XML document for reading, prevent other instances from writing to the document while I'm reading it, and then write to the document before allowing other instances to do the same thing?"

In this case, you should handle opening and closing the file yourself rather than allowing TXMLDocument to do it for you. Use a TFileStream to open the file with an exclusive read and write lock and XMLDocument.LoadFromStream instead of LoadFromFile. Save the document with SaveToStream after resetting the stream.Position to 0. Use a try/finally in order to ensure that you close the stream when you are done with it. Since you're exclusively locking the file, you no longer need the temp file or any other kind of mutex.

Obviously, opening the file could fail if another instance is currently reading/writing to it. So you need to handle this and retry later on.

Answer 4

Just remember that every time you need to add a node, the entire document must be reloaded and reparsed. Depending on the size of the XML document and what data you are saving, it might not be the most efficient method of transferring data.

The approach of writing to a separate file is an interesting solution, one to consider would be to have your "multiple instance" apps write unique XML files and then load those into a master document with a separate program using a FindFirst loop. That way you can keep your xml structure pretty much intact without any major changes to your existing programs.

Answer 5

From this answer:

On Windows, this is possible if you can control both programs. LockFileEx. For reads, open a shared lock on the lockfile. For writes, open an exclusive lock on the lockfile. Locking is weird in Windows, so I recommend using a separate lock file for this.

('Both programs' dos not apply in your case at it is the same program, just running in multiple instances.)

Side note / how I found this answer: The Java logging library logback uses the platform specific file locking API (via NIO) to implement the 'prudent mode' where multiple processes can log into the same file without corrupting it - something which iiuc is not possible with the Delphi RTL file operations.