Category Archives: Windows 8

Notes on Windows 8 VHDX file format

November 16, 2011 – 1:16 am

Last year, I wrote an article on Technet regarding the performance issues with the dynamic VHD file format. That article can be fund here . The main point made in the article was that the dynamic VHD file format ensured that 3 out of every 4 2MB data blocks were misaligned in the sense that they did not begin on a 4096 byte boundary. With the future of large sector disks, this will become even more important.

With Windows 8, Microsoft appears to have addressed this issue with the new VHDX file format. As disclosed in some talks at the BUILD conference, the VHDX file format

  • Extends VHD file based volumes to 16TB from the current 2TB limit
  • Improves performance
  • Makes the file format more resilient to corruption

While the best way to evaluate these features would be to compare the VHDX file format with the VHD file format, unfortunately Microsoft has not yet released the VHDX file format.

I decided to look into this a little further and wrote a little applet that writes 4kb at offset zero in a file, and 4kb at offset 2MB in a file. I ran the program twice

  • Once with the file housed in a VHD based volume
  • Once with the file housed in a VHDX based volume

 Given that the VHD file format in Windows Server 2008 R2 uses 2MB data blocks, the applet effectively writes at the beginning of the first 2 data blocks.

While I plan to analyze the I/Os in more detail, for now, there is one interesting observation. Here are the I/Os on the VHD file traced using Process Monitor in the Windows Server 2008 R2 parent.

And here are the I/Os traced with the file housed in a VHDX based volume

The immediate observation is that there are some 512 byte writes on the VHD based volume whereas the VHDX based volume shows no 512 byte writes at all. The 512 byte writes are presumably the writes to the Sector bitmap of the VHD (file format). While the conclusion is not definitive, one is drawn to believe that the 512 byte sector bitmap has been replaced and/or moved – maybe the sector bitmaps are now all together and not interspersed between data blocks.

More on this topic in a later blog.

Advertisement
By dilipcnaik | Also posted in Hyper-V, NTFS | Comments (0)

SMB 2.2 and other Windows 8 protocol documentation

October 7, 2011 – 2:08 am

With Windows 8, Microsoft is significantly pushing the envelope on Windows Server performance and high availability with Windows 8, while also enabling these features with low cost Network Attached Storage.

Recall the days when Microsoft was fined 2 million odd dollars per day for not publishing proper documentation for various protocols. What is remarkable is that while Windows 8 has not even reached a Beta stage, the documentation for the various different new protocols in Windows 8 has already been published at this MSDN link. In no particular order, here is a summary of the various storage and NAS protocols you will find at that MSDN link

  1. Windows 8 will support Hyper-V and SQL storing VHD files on an SMB 2.2 NAS share. A new File Server Remote VSS Protocol will enable shadow copy based backups (and restores) of these NAS shares.
  2. The new SMB 2.2 protocol that
    1. Allows clients to obtain leases on directories and not just files
    2. Extensions to the SMB 2 protocol that allow for a client to form multi-channel based multiple connections to a server and have the data flow in parallel across all channels, thus providing bandwidth aggregation as well resiliency by dropping down to remaining existing channels when a given channel fails
    3. Allows clients to retrieve hashes for ranges of a file – this SMB is used in branch cache scenarios
    4. A new SMB over RDMA protocol that allows for setting up and SMB 2 based client/server connection and the transferring data using an RDMA capable transport such as iWarp or Infiniband
    5. A Storage Services protocol that provides for scenarios such as creating modifying volumes and shares, creating and managing shadow copies, etc.

Notably absent so far is a new Witness Notification Protocol document that forms the basis for high availability NAS shares and is the basis of the highly significant enhancements to have clients transparently fail over to new servers/shares as needed, without needing to re-open files. Presumably, this document and other remaining documents will get posted soon.

Note that all of the documents are marked preliminary. This is not surprising, given that Windows 8 has not yet reached a Beta stage, and presumably Microsoft reserves the right to make modifications as needed

By dilipcnaik | Also posted in SMB 2.2 | Comments (0)

NTFS improvements for high availability in Windows 8

October 6, 2011 – 11:07 pm

Caveat: While efforts are made to make the content accurate, you peruse the contents of this blog at your own risk and responsibility. Any actions you take, or do not take based upon the blog contents are also your own responsibility.

Microsoft has strived to improve NTFS resiliency and availability through the various versions of Windows. This post focusses on one aspect: the availability of an NTFS volume when corruption is detected.

Prior to Windows Server 2008 R2, when an NTFS volume corruption is detected, the volume is taken offline and a process known as Chkdsk run to fix the corruption. The amount of time Chkdsk took to run was somewhat proportional to the number of files and directories within the NTFS volume. Thus smarter administrators sized NTFS volumes in a way that limited the maximum amount of files and directories within a given volume. For some corporations, this meant additional NTFS volumes with the associated management overhead. Starting with Windows NT4 and up to Windows 2008, Microsoft invested in efforts to reduce the amount of time it took to run Chkdsk.

With Windows Server 2008 R2, Microsoft changed tack and focused on improving NTFS volume availability by:

  1. Further reducing the amount of time it took Chkdsk to run. In particular, this was done by reading NTFS meta data in bigger chunks and by better caching it, thus avoiding disk IOPS. Note that most of the time spent in Chkdsk is scanning the volume to detect issues, rather than in actually fixing the issues. This is where the time taken to run Chkdsk being proportional to the number of files & directories arises from.
  2. Reducing the frequency with which Chkdsk needed to run by better detection of whether the volume was corrupted or not
  3. Fixing at least some of the corruption issues without taking the volume off line, a process Microsoft termed “self healing”

Windows 8 takes this logical progression a step further

  1. Scanning to detect errors is done while the volume is online and available.
  2. As in step 3 above, self healing fixes some of the corruption issues. Indications are that a further set of errors is fixed by self healing in Windows 8 as compared to prior versions of Windows
  3. The errors that cannot be corrected are logged
  4. The volume is taken offline and the errors identified in phase C are corrected. This implies the volume is unavailable only for the duration taken to fix the errors logged in phase C.
  5. In rare instances, the scanning determines that the corruption is too serious and needs a full offline scan, just as in Windows Server 2008 R2 and earlier versions of Windows.

Diagram 1 is a slide from a talk at the Windows 8 BUILD conference and the whole slide deck is available at this link

The item to note from the diagram is that the amount of time taken to fix a corrupted volume is now independent of the number of files and directories within the volume with one caveat. The caveat is that the corruption is not of an extremely serious nature – those still require the old style full scan while the volume is unavailable.

Microsoft now supports NTFS volumes of up to 24TB in size with Windows 8 and effectively, the number of files and directories within a given volume may also be increased without fear of the volume being unavailable for a long time due to corruption.

The changes in NTFS imply that an NTFS volume has more states in Windows 8 compared to prior versions of Windows. Tables 1 and 2 summarize the NTFS volume states in Windows 8 and prior versions of Windows

Table 1 Windows Server 2008R2/Windows 7 and earlier version of Windows NTFS volume states

NTFS Volume State 2k8R2/Win7 & earlier versions of Windows Description
Clean/Healthy “All is well” – no corrective action needed
Dirty/Potentially needs repair Can result even from an incorrect shutdown – indicates a scan and potential fixing needed

 Table 2 Windows 8 NTFS Volume states

Windows 8 NTFS volume states Description
Clean/Healthy “All is well” – no corrective action needed
Scan Needed/Dirty Indicates scan and potential fixing needed, but the scan will be done while the volume is mounted
Spotfix needed Scan has identified the errors, some were “self healed”, some marked for quick offline fixing, the volume needs to be briefly taken offline and the logged errors fixed
Full Chkdsk needed  The errors are too serious to spot fix and an old style full scan with the volume being offline for a longer time is needed. Presumably this happens only rarely.

 Note that NTFS volumes are still “compatible” – you can take an NTFS volume created with prior versions of Windows and mount it on a Windows 8 system. You can also take an NTFS volume created with Windows 8 and use it with earlier versions of Windows.

By dilipcnaik | Also posted in NTFS | Comments (1)