NTFS improvements for high availability in Windows 8

Caveat: While efforts are made to make the content accurate, you peruse the contents of this blog at your own risk and responsibility. Any actions you take, or do not take based upon the blog contents are also your own responsibility.

Microsoft has strived to improve NTFS resiliency and availability through the various versions of Windows. This post focusses on one aspect: the availability of an NTFS volume when corruption is detected.

Prior to Windows Server 2008 R2, when an NTFS volume corruption is detected, the volume is taken offline and a process known as Chkdsk run to fix the corruption. The amount of time Chkdsk took to run was somewhat proportional to the number of files and directories within the NTFS volume. Thus smarter administrators sized NTFS volumes in a way that limited the maximum amount of files and directories within a given volume. For some corporations, this meant additional NTFS volumes with the associated management overhead. Starting with Windows NT4 and up to Windows 2008, Microsoft invested in efforts to reduce the amount of time it took to run Chkdsk.

With Windows Server 2008 R2, Microsoft changed tack and focused on improving NTFS volume availability by:

  1. Further reducing the amount of time it took Chkdsk to run. In particular, this was done by reading NTFS meta data in bigger chunks and by better caching it, thus avoiding disk IOPS. Note that most of the time spent in Chkdsk is scanning the volume to detect issues, rather than in actually fixing the issues. This is where the time taken to run Chkdsk being proportional to the number of files & directories arises from.
  2. Reducing the frequency with which Chkdsk needed to run by better detection of whether the volume was corrupted or not
  3. Fixing at least some of the corruption issues without taking the volume off line, a process Microsoft termed “self healing

Windows 8 takes this logical progression a step further

  1. Scanning to detect errors is done while the volume is online and available.
  2. As in step 3 above, self healing fixes some of the corruption issues. Indications are that a further set of errors is fixed by self healing in Windows 8 as compared to prior versions of Windows
  3. The errors that cannot be corrected are logged
  4. The volume is taken offline and the errors identified in phase C are corrected. This implies the volume is unavailable only for the duration taken to fix the errors logged in phase C.
  5. In rare instances, the scanning determines that the corruption is too serious and needs a full offline scan, just as in Windows Server 2008 R2 and earlier versions of Windows.

Diagram 1 is a slide from a talk at the Windows 8 BUILD conference and the whole slide deck is available at this link

The item to note from the diagram is that the amount of time taken to fix a corrupted volume is now independent of the number of files and directories within the volume with one caveat. The caveat is that the corruption is not of an extremely serious nature – those still require the old style full scan while the volume is unavailable.

Microsoft now supports NTFS volumes of up to 24TB in size with Windows 8 and effectively, the number of files and directories within a given volume may also be increased without fear of the volume being unavailable for a long time due to corruption.

The changes in NTFS imply that an NTFS volume has more states in Windows 8 compared to prior versions of Windows. Tables 1 and 2 summarize the NTFS volume states in Windows 8 and prior versions of Windows

Table 1 Windows Server 2008R2/Windows 7 and earlier version of Windows NTFS volume states

NTFS Volume State 2k8R2/Win7 & earlier versions of Windows Description
Clean/Healthy “All is well” – no corrective action needed
Dirty/Potentially needs repair Can result even from an incorrect shutdown – indicates a scan and potential fixing needed

 Table 2 Windows 8 NTFS Volume states

Windows 8 NTFS volume states Description
Clean/Healthy “All is well” – no corrective action needed
Scan Needed/Dirty Indicates scan and potential fixing needed, but the scan will be done while the volume is mounted
Spotfix needed Scan has identified the errors, some were “self healed”, some marked for quick offline fixing, the volume needs to be briefly taken offline and the logged errors fixed
Full Chkdsk needed  The errors are too serious to spot fix and an old style full scan with the volume being offline for a longer time is needed. Presumably this happens only rarely.

 Note that NTFS volumes are still “compatible” – you can take an NTFS volume created with prior versions of Windows and mount it on a Windows 8 system. You can also take an NTFS volume created with Windows 8 and use it with earlier versions of Windows.

Advertisements
Post a comment or leave a trackback: Trackback URL.

Comments

  • Monts  On November 23, 2011 at 2:24 am

    Interesting blog Dillip found it via goggling for my issue … now I have been testing the win 8 dev build and I also use Linux (Fedora) I am very comfortable using both O.S
    My setup is somewhat like this… a 1 TB hdd ntfs formatted dynamic volume (used as a backup/data drive) another disk of 250 gb single basic ntfs volume has win 8 installed … the third hdd a 500gb has Linux installed.

    A few days back my Linux disk developed few bad sectors and the data volume kept giving read errors … so from withing Linux I copied data to the windows 8 disk … but after booting into windows it showed errors .. after a disk check most of the copied data was deleted or garbled… now in the past I have use Linux to write data to ntfs volumes with no problems at all.

    The 1tb disk has very less space left .. plus now it’s a dynamic volume so it can’t be used with standard Linux tools.. that is the reason I am trying to use the win 8 volume as a backup drive. What I strongly suspect is that MS has implemented some type of data verification module or fs filter to monitor modification outside the windows 8 environment.

    That is why the filesystem is being marked dirty and a disk check done (which is deleting the files copied). Under Linux they show copy completed with no issue. I can even read those files from Linux on the ntfs drive.

    Would you care to comment on this issue ?? Thanks.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: