How to ensure all data has been physically written to disk?

StackOverflow https://stackoverflow.com/questions/383324

  •  23-08-2019
  •  | 
  •  

Question

I understand that .NET FileStream's Flush method only writes the current buffer to disk, but dependent on Windows' disk driver and the hard disk firmware this is no guarantee that the data is actually physically written to disk.

Is there a .NET or Win32 method that can give me this guarantee? So if there is power loss one nanosecond after the call to this method comes back, I can still be sure that everything is OK?

Was it helpful?

Solution

Under Windows, look at FlushFileBuffers (Win32 API).

OTHER TIPS

Stefan S. said:

I understand that .NET FileStream's Flush method only writes the current buffer to disk

No, .NET FileStream's Flush only writes the .NET buffers to the OS cache, it does not flush the OS cache to disk. Sadly the MSDN doc on this class doesn't say that. For .NET < 4.0, you'll have to call Flush + Win32's FlushFilebuffers:

using System.Runtime.InteropServices;
. . .

// start of class:
[DllImport("kernel32", SetLastError=true)]
private static extern bool FlushFileBuffers(IntPtr handle);
. . .

stream.Flush();     // Flush .NET buffers to OS file cache.
#pragma warning disable 618,612 // disable stream.Handle deprecation warning.
if (!FlushFileBuffers(stream.Handle))   // Flush OS file cache to disk.
#pragma warning restore 618,612
{
  Int32 err = Marshal.GetLastWin32Error();
  throw new Win32Exception(err, "Win32 FlushFileBuffers returned error for " + stream.Name);
}

For .NET 4.0, you can instead use the new flush(true) method. 11/09/2012 update: MS bug report here says it's broken, then fixed, but doesn't say what version or service pack it was fixed in! Sounds like bug was if internal .NET FileStream buffer is empty, the Flush(true) did nothing??

Well, you could close the file... that would probably do it. In reality, with HAL abstraction, virtualization, and disk hardware now having more processing power and cache memory than computers did a few years ago, you're going to have to live with hoping the disk does its job.

The transactional file system never really materialized ;-p Of course, you could perhaps look at using a database as a back end, and use the transaction system of that?

Aside: note that not all streams even guarantee to Flush() - for example, GZipStream etc retain a working buffer of uncommitted data even after a flush - the only way to get it to flush everything is to Close() it.

I've noticed that the .NET 4 #Flush(true) doesn't actually write to the disk. We were having strange issues with corrupted data and I found this bug report on the MS site:

The details tab for the bug report has a test program you can run that will show the issue;

  1. Write a bunch of data to disk
  2. fs.Flush(true). This takes no time (much faster than can possibly written to the disk).
  3. Use the win32 API FlushFileBuffers. This takes a long time.

I'm changing over to the win32 FlushFileBuffers call...

The file data that's buffered in the file system cache to be written to disk. That data is normally lazily written, based on the position of the disk write head. Having a gigabyte of cached data is technically possible so it can take quite a while. If this is important to you then consider the FileOptions.WriteThrough option instead.

There is a simple answer to flushing the content of the buffer to disk. After your WriteAllText function, open file, close it, and reset it

here is an example

My.Computer.FileSystem.WriteAllText(yourfilename, "hello", False, System.Text.Encoding.ASCII)
FileOpen(1, yourfilename, OpenMode.Input)
FileClose(1)
Reset()

There's simply too many levels of abstraction to be absolutely sure that the data is written to the disc, right down to the hardware level.

Not brilliantly performant or foolproof, but how about re-opening the file once it is written in a seperate process and checking the size or contents?

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top