NTFS volume defragmentation – Part 1 – a developer perspective

This blog is about NTFS volume fragmentation from a developer perspective.

As a developer, my perspective is that many applets, including, but not limited to Microsoft tools and utilities, provide NTFS with insufficient information to place a file on a volume so as to certainly avoid defragmentation.

As an example, assume we are coding a file copy applet. The typical applet, with some oversimplification, might look something like this

Open source file

Open destination file

While (!EndOfSourceFile)

{

                Read(SourceFile)

                CheckForEndOfFile

                WriteToDestinationFile (including write a partial buffer if any)

}

Close source file

Close destination file

The example ignores the code to get and set file attributes and ACLs, purely to concentrate on the fragmentation that occurs while writing the default data stream.

Assume both files are buffered. At some point in time – and this point is typically non deterministic – it depends upon system state including what else is in the Cache Manager buffers, what other apps are running etc. the Cache Manager will decide to flush the destination file. At this point in time, the Cache Manager will have N buffers cached, with N being a number depending upon system state. If you copy the same file one more time, N may be the same, or may be different, when the Cache Manager decides to flush the file buffers during that second copy.

Now consider what NTFS needs to do. It needs to decide where to place the destination file, and write out the buffers being flushed. In effect, the copy applet we wrote is telling NTFS:

  1. Find a place for the file on the volume
  2. I am not telling you how big the file is, but make sure it does not cause fragmentation!

Unconfirmed rumors are that NTFS guesses the file size to be twice the size of the Cache Manager contents and based on that assumption, places the file on the volume. Whether that is true or not is immaterial.

The takeaway is that NTFS needs to know the size of a file before it decides where to optimally place the file – and very often, that information about the file size is unavailable when NTFS is asked to make that file placement decision.

A future blog will explore this further and examine what APIs an application can invoke to provide NTFS with all the information it needs to avoid fragmentation.

Advertisements
Post a comment or leave a trackback: Trackback URL.

Comments

  • Ayush Gupta  On November 21, 2011 at 3:51 pm

    I think a good way for the copy apps to avoid fragmentation is to allocate space to the file during its creation (NtCreateFile). CreateFile API does not provide this. Maybe SetEndOfFile should be used there. And then start copying data.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: