Monthly Archives: July 2012

Aligned I/O and large sector disks – VHD, VHDX, and VS 2012 install

July 28, 2012 – 7:10 pm

The issues of aligned disk I/O have been understood for a while. One example is where the Microsoft VHD file format ensured that ¾ of data blocks within a VHD file would be aligned on a 512 byte boundary, and not a 4096 sector boundary. See the Technet blog I authored a few years ago “Some observations on Dynamic VHD Performance”.

That makes using VHD files with large sector drives (which read and write data in 4k units) extremely painful. See the MSDN KB article “Using Hyper-V with large sector drives on Windows Server 2008 and Windows Server 2008 R2”. The KB article mentions a performance degradation of up to 30%.

The problem here is that Hyper-V performs non-cached I/Os which result in a Read, Modify, Write cycle as the KB article describes. For example, if 4kb of data needs to be written, and if that data spans 2 adjacent physical 4kb sectors, Windows needs to read 8kb, modify 4kb of this 8kb data, and then write back 8kb of data.

Recognizing the problem, the Hyper-V team introduced the VHDX file format in Windows Server 2012. Look at a fellow MVP blog “Why Windows Server Hyper-V VHDX 4k alignment is so important”

And now, let us return to my earlier blog about Visual Studio (VS) 2012 installer performing non-aligned writes. Since these are cached writes, the performance degradation on large sector disks will not be as noticeable. But just because one issues cached writes does not mean the data stays in cache until you are done writing to the file and close the file, when the cache is flushed. Since Visual Studio writes in a pattern where the first and last writes to a file are odd length bytes, and all intermediate writes are even bytes, the data in the cache, while the file is being written to, is always odd bytes in length. So should the cache get flushed, for any reason, you now have the equivalent of misaligned I/Os, and the penalty on large sector disks is even more noticeable.

I will be looking to buy a large sector disk and try installing VS 2012 to that disk. In the meanwhile, I would love to hear from any reader that has already tried this experiment.

Advertisement
By dilipcnaik | Posted in Uncategorized | Comments (1)

The perils of alignment for memory access and disk I/O

July 28, 2012 – 6:30 pm

In my earlier blog, I described how Visual Studio (VS) 2012 is now a requirement for writing kernel mode drivers on both the x86/x64 Intel/AMD, and also the ARM version of Windows 8. So I installed VS 2012 RC on two different laptops and was unhappy with the installation time. I must place on my record my appreciation for the Visual Studio team, which has been very diligent in following up and looking into the issue. Of course, I will acknowledge that my belief of “it takes too long” could be incorrect, and I may be encountering unusual circumstances on both my systems. So with that caveat that perhaps “I am encountering a one off situation”, here we go with my analysis.
First, a couple of references are in order.

  1.  To quote from MSDN “In this document we explain why you should care about data alignment, the costs if you do not, how to get your data aligned, and what to do when you cannot. You will never look at your data access the same way again.” The point is; aligned memory access in Windows is very important.
  2. It is equally important to ensure that writes are aligned as well. Most current disks write data in 512 chunks called sectors. So if you write 512 bytes at offset zero, a single write suffices. But if you write 512 bytes at offset 1, the I/O spans 2 disk sectors. So a single write becomes read 2 disk sectors, copy over the new 512 bytes of data, and issue 2 sector writes, each of 512 bytes. So a 512 byte write becomes a 1024 byte read and a 1024 byte write. Here is an MSDN blog explaining among other things, the importance of aligned I/Os for SQL. And here is a another MSDN SQL blog explaining the importance of aligned I/O

Now back to the topic at hand – installing Visual Studio 2012 RC and analyzing possible causes for why it takes as long as it does. So I decided to investigate further, by tracing the I/Os using Sysinternals (now part of MSDN) tool Process Monitor.

Here is a screen shot showing a small part of the I/O of the installation. Note that I randomly located this I/O pattern. I also cursorily checked that other files have similar behavior; in particular, write an odd number of bytes at offset zero, and then proceed to write the rest of the file.

Image

For file DataCollection.dll, please notice

  1. The write at offset zero for 32,447 bytes
  2. The write at offset 32,447 for 32,768 bytes
  3. The write at offset 62,215 for 16761 bytes
  4. The total file size is 81,976 bytes and 32,447 + 32,768 + 16,761 = 81976

Now apply the logic of the references quoted – in particular, the importance of aligned memory access, and aligned disk I/O access.

At the very least, the each of the 3 I/Os will consist of a 1 or 3 byte copy, a copy of some N DWORDs, followed by a 1 or 3 byte copy. This could have been completely avoided by doing 3 I/Os, each consisting of an even number of bytes. There is a penalty to be paid for the 1 byte and 3 byte memory access.

I must admit that this trace is at the file system layer. It is certain that before the I/O hits the disk, which is a block mode I/O, the Windows Cache Manager and I/O subsystem will have intervened to make the I/O aligned and an integral number of sectors. There will still be some disk I/O penalties however, when some writes get split across 2 adjacent sectors. This could be avoided. Consider the case where say part of the file has been written, and is in cache. And the I/O pattern guarantees that there will be an odd number of bytes cached, until the final odd length write arrives. Now imagine that for some reason, the cache gets flushed before the last write arrives. This could be because the file is very large, or there is memory pressure. This means that the cache manager will zero fill a buffer until the end of a sector (an odd number of bytes) and then write out that sector. When the next write arrives, this just flushed sector needs to be read, the zero filled bytes are copied over with the newly arrived data, and then the same sector is written – again!

There is no perceivable advantage in making the I/O nonaligned – and significant potential harm. It is difficult to estimate how much VS 2012 installation will speed up, were the writes to be aligned.
There are other oddities as well in the trace, but I will write about those in future blogs.

I invite reader comments on whether they believe this I/O pattern is within acceptable bounds. For readers willing to trace their VS 2012 install, I would also welcome feedback as to whether they observe this pattern.

By dilipcnaik | Posted in Hyper-V, NTFS, Uncategorized, VHDX, Windows 8 | Comments (0)

Developing kernel mode drivers for Windows 8

July 17, 2012 – 7:30 pm

I have been developing Windows kernel mode drivers for 10+ years now and notice that the Windows 8/Windows Server 2012 WDK brings some changes. This blog tries to highlight the changes in the hope that other developers will benefit.

I went through the mechanics of installing the Windows 8 WDK on 2 different laptops. So with the caveat “Your mileage may vary – maybe I hit the jackpot and my experience was unique”, here we go:

  1. WDK 8 requires that you first install Visual Studio 2012. See http://msdn.microsoft.com/en-US/windows/hardware/hh852362 and the listed System Requirements section that among other things, state “Before you begin, you must first install Visual Studio Professional 2012 RC or above”
  2. For now, Visual Studio 2012 is “free” since it is not yet a released product. Presumably, two different parts of Microsoft will soon tell us a couple of important data points
    1. The Visual Studio team will tell us pricing for the various different versions/SKUs of Visual Studio 2012
    2. A different part of Microsoft will tell us which SKUs are acceptable for compiling the WDK code samples and code developers write
    3. I am not referring to the new ARM based version of Windows called WindowsRT. I am referring to writing drivers for the x86/X64 platform. Even that one now requires Visual Studio 2012.
    4. Visual Studio is an excellent product for whom it works. A while back – as in 5 or so years ago, I abandoned it, primarily due to the long install time and resources it consumed in terms of disk space. The only use I had for it was the compiler. I use a different editor, and I use WinDbg in stand alone mode.  So when a previous version of the WDK (called the DDK at that time) shipped with compilers, I abandoned Visual Studio. I don’t seem to have the same choice any more.
    5. Depending upon which version of Visual Studio you install, and depending upon what choices you make during the installation, Visual Studio will take some time to install and occupy some GBs (certainly less than 10GB) of disk space.
    6. In case you are still reading, the WDK no longer downloads sample code. My gratitude to the people who posted this fact e.g. http://boardreader.com/thread/Samples_arent_installed_along_with_the_W_u8jjs__3e9c9b67-ea9f-4225-a268-5d5ece555568.html Presumably, this makes it easier for Microsoft to release new or updated code samples without shipping the whole WDK
    7.  The code samples are available at http://code.msdn.microsoft.com/windowshardware . Presumably Microsoft will release some scripts to download the samples in some sort of collection e.g. all the samples, all the USB samples, all the storage driver samples, etc. But meanwhile, one has to download the samples one sample at a time. While this saves on disk space, the savings are miniscule compared to the added GBs occupied by VS 2012.
    8. I must acknowledge that VS 2012 now provides an ARM capable compiler, something the old WDK did not.

We will have to wait and see what the VS 2012 requirement adds in terms of software licensing costs. I guess that is just “the cost of doing business” with Windows 8.

In the meanwhile, I look forward to attending the next plug fest and testing my driver(s) for CSV 2.0 compatibility.

By dilipcnaik | Posted in NTFS, SMB, Windows 8 | Comments (0)