why fsutil.exe takes less time to write a huge file into disk than programmatically?

Question

this question is according to this topic: creating a huge dummy file in a matter of seconds in c#

I just checked the fsutil.exe in xp/vista/seven to write a huge amount of dummy data into storage disk and it takes less time to write such a big file in comparison to programmaticly way.

When I'm trying to do the same thing with the help of .net it will take considerably more time than fsutil.exe.

note: I know that .net don't use native code because of that I just checked this issue with native api too like following:

long int size = DiskFree('L' - 64);
const char* full = "fulldisk.dsk";
__try{
Application->ProcessMessages();
HANDLE hf = CreateFile(full,
                       GENERIC_WRITE,
                       0,
                       0,
                       CREATE_ALWAYS,
                       0,
                       0);
SetFilePointer(hf, size, 0, FILE_BEGIN);
SetEndOfFile(hf);
CloseHandle(hf);
}__finally{
    ShowMessage("Finished");
    exit(0);

and the answer was as equal as .net results.

but with the help of fsutil.exe it only takes less duration than above or .net approaches say it is 2 times faster

example : for writing 400mb with .net it will take ~40 secs the same amount with fsutil.exe will take around 20secs or less.

is there any explanation about that? or which function fsutil.exe does use which has this significance speed to write?

Answer 1

I don't know exactly what fsutil is doing, but I do know of two ways to write a large file that are faster than what you've done above (or seeking to the length you want and writing a zero, which has the same result).

The problem with those approaches is that they zero-fill the file at the time you do the write.

You can avoid the zero-fill by either:

1. Creating a sparse file. The size is marked where you want it, but the data doesn't actually exist on disk until you write it. All reads of the unwritten areas will return zeros.
2. Using the SetFileValidData function to set the valid data length without zeroing the file first. However, due to potential security issues, this command requires elevated permissions.

Answer 2

I agree with the last comment. I was experimenting with a minifilter driver and was capturing IRP_MJ_WRITE IRPs in callback. When I create or write to the file from cmd line or win32 application, I can see writes coming down. But when I created a file using "fsutil file createnew ..." command, I don't see any writes. I'm seeing this behavior on win2k8 r2 on NTFS volume. And I don't think (not sure 100% though) it is a sparse file either. It is probably setting the size properties in MFT without allocating any cluster. fsutil do check the available free space, so if the file size is bigger than free space on disk, you get error 1.

I also ran program sening FSCTL_GET_RETRIEVAL_POINTERS to the file and I got one extent for the entire size of the file. But I believe it is getting all data

Answer 3

This is something that could be

• Written in Assembler (Raw blinding speed)
• A Native C/C++ code to do this
• Possibly an undocumented system call to do this or some trick that is not documented anywhere.

The above three points could have a huge significant factor - when you think about it, when a .NET code is loaded, it gets jit'ted by the runtime (Ok, the time factor would not be noticeable if you have a blazing fast machine - on a lowly end pentium, it would be noticeable, sluggish loading).

More than likely it could have been written in either C/C++. It may surprise you if it was written in Assembler.

You can check this for yourself - look at the file size of the executable and compare it to a .NET's executable. You might argue that the file is compressed, which I doubt it would be and therefore be inclined to rule this out, Microsoft wouldn't go that far I reckon with the compressing executables business.

Hope this answers your question, Best regards, Tom.

Answer 4

fsutil is only fast on NTFS and exFAT, not on FAT32, FAT16 This is because some file system have an "initialized size" concespt and thus support fast file initialization. This just reserves the clusters but does not zero them out, because it notes in the file system that no data was written to the file and valid reads would all return 00-filled buffers.