Category Archives: SMB

SMB 3 NAS is preferable to DAS in a Windows environment

July 28, 2014 – 2:44 am

Microsoft is investing heavily in the Network Attached Storage (NAS) protocol SMB 3 and is clearly laying out a road map that suggests NAS is the future as opposed to Direct Attached Storage (DAS). Consider:

  • SQL Server 2012 system d/b and user d/bs, as well as Hyper-V 2012 workloads can be placed on NAS provided the NAS is SMB 3!
  • Microsoft made significant speed improvements in the SMB 3 client and server to have NAS achieve 97% of the speed of DAS, and this is without hardware acceleration.
  • Microsoft invested in SMB 3 Multi Channel by aggregating the bandwidth using parallel TCP channels using multiple NICs at the SMB 3 protocol layer. Multi Channel is all about speed AND reliability where failed I/Os are seamlessly moved to a different TCP channel when one channel fails.
  • Continuing on the speed theme, Microsoft invested in RDMA support via SMB Direct, which requires not just SMB 3, but also SMB 3 Multi Channel. The maximum IOPS on a Windows system is achieved when using SMB 3 NAS with SMB Direct support, NOT with DAS!
  • Going back to the reliability theme, SMB 3 includes support for Persistent Handles, which combined with the Witness Protocol, ensure applications such as SQL, Exchange, and Hyper-V never see an I/O failure, and the I/O is seamlessly moved to a different node as needed. This only works with SMB 3 NAS, and does NOT work with DAS!
  • I have been asked numerous times “But Microsoft has invested in Storage Spaces and Tiering where data is moved between SSD and spinning media to optimize performance. Does that not indicate Microsoft advocates DAS?” And my answer has always been “Storage Spaces is even more valuable when used as the storage backing a Windows Server 2012/R2 NAS!” Using Storage Spaces does not mean one has to abandon NAS.
  • Microsoft supports deduplication of VDI VMs, but the only supported configuration is with the VDI VM files residing on an SMB 3 based Windows Server 2012 R2 based NAS! (and not with DAS!)
  • To provide examples of other Microsoft efforts leveraging SMB3 , consider the simple “copy” or “xcopy” command to say copy a GBs large file. Microsoft changed the CopyFileEx API to leverage all SMB 3 features including SMB 3 credits, SMB 3 Multi Channel, and SMB Direct (RDMA) to ensure the file copy is as fast as possible.
  • The Microsoft Hyper-V team re-wrote live migration in Hyper-V 2012 R2 to leverage SMB 3. While migrating a VM, Hyper-V 2012 setup its own TCP channel to copy the VM RAM. Hyper-V 2012 R2 uses SMB 3, and thereby gets the speed/reliability improvements of SMB 3 while doing the same copy.
Advertisement
By dilipcnaik | Also posted in Hyper-V, NTFS, SMB 2.2, SMB 3.0, VHDX, Windows 8, Windows Server 2012 | Comments (0)

Protocol Converter between CIFS, SMB2, SMB3, and NFS

May 12, 2014 – 5:11 am

As a Microsoft Storage MVP, I am always looking for ways to fill in the gaps between what Windows natively offers, and what seems to be useful for enterprise and consumer scenarios.

One intriguing product idea that I have built and have an advanced prototype running is a “Protocol Converter”. I am open to different names for the product since it really does not do justice to the myriad of use cases I can see. And I am sure some readers will point out even more use cases than I am missing at the moment. This is the first of a series of planned blogs around this “Protocol Converter” idea.

The “product requirements” as I set them include:

  • Be able to freely convert any of CIFS, SMB2, SMB3, NFS to any of the other protocols. So in particular, be able to do all of these conversions

o   CIFS <-> SMB2, CIFS <->SMB3 CIFS <->NFS

o   SMB2 <-> CIFS, SMB 2 <-> SMB3, SMB2 <-> NFS

o   SMB3 <-> CIFS, SMB3 <-> SMB2, SMB3<->NFS

o   NFS <-> CIFS, NFS <-> SMB2, NFS <-> SMB3

  • Develop this product with minimal resources
  • Develop a highly maintainable product
  • Develop a product with a very high probability of working with future protocol revisions such as SMB 3.1 or SMB 4.0 (imagined names). Of course, some testing and development may be needed depending upon what features these unknown protocols will have.
  • Have an enterprise ready product, but of course, even such products begin life as a prototype

Slide1

 

Figure 1 summarizes what this conceptual Protocol Converter looks like.

After reflecting on the product requirements for a while, I decided to write as little protocol specific code as possible. I have spent years developing CIFS & SMB 2/SMB 3 stacks, and while this work has been enjoyable, I decided the world does not need yet another implementation of any of CIFS, NFS, SMB 2 or SMB 3 protocol stacks. BTW I am extremely happy about the awesome SMB2/SMb 3 protocol stack that we have developed at www.HvNAS.com . It runs on any Linux/Unix and any CPU including Intel x86 and little endian CPUs. But back to the Protocol Converter, where a key observation is that Windows Server 2012 (and 2012 R2) already ship with all the protocol parsers I am looking for, both on the client and the server side! And that is what the prototype code leverages – as of now, it has zero protocol specific code!

I see three main phases in terms of code development for this project:

  • Develop a “data path solution” where all data I/O for all protocol conversions works. So things such as file creation, deletion, enumeration, read, write, etc. work. This piece is already working, though it needs more testing.
  • Develop a security solution that enforces enterprise class access control across multiple domains, etc. The “data path” solution does not enforce proper access control, but then again, this is just a product development milestone and not a shipping product yet!
  • Add some protocol specific features that deal with differences between protocols e.g. deal with oplocks that exist on one side of the Protocol Converter, but not the other

I will write blogs to track progress on these additional development tasks as well as what I perceive to be use cases for this Protocol Converter.

I welcome any potential beta testers for this product.


By dilipcnaik | Also posted in Hyper-V, NTFS, SMB 2.2, SMB 3.0 | Comments (1)

Backup performance and SMB 3 Multi Channel

March 3, 2014 – 8:30 pm

In this day and age of exploding data amounts, backup and restore is both increasingly important, and becoming more common and taken for granted. But not all backup “target systems” i.e. the systems to which data is backed up are created equal. Especially so, when the system being backed up is Windows based.

  1.  If your backup target system is based upon CIFS (also sometimes referred to as SMB 1), backup (and restore) is limited to 64kb serial I/O. In other words, the backup/restore software does a 64kb I/O, waits for the I/O to complete, and only then issues the next I/O. In fact it is worse than this. The total payload is limited to 64kb and hence well behaved apps that want to perform I/O in 4MB block size will only use a 60kb payload (data).
  2. If your backup target system is running SMB 2.0, the I/O is 1MB serial, which is certainly an improvement.
  3. If your backup target is SMB 2.1, the I/O is again 1MB, but SMB 2.1 has a server issuing multiple credits which means the client can issue multiple I/Os without having to wait for any one of the I/Os to complete. A typical Windows to Windows flow will show 10 1MB I/Os on the wire at the same time. Note that this is all on a single TCP channel. So the backup/restore speed is significantly higher
  4. Now recall that in most cases, BOTH the system being backed up AND the backup target are servers. For example, you could be backing up a file server or SQL server or Hyper-V server, and of course, the backup target also operates typically as a NAS (file server).  Thus it is very likely that at least one of the two has multiple NICs. If any one (or both) ends of an SMB 3 connection have multiple NICs, and provided these NICs are 10GB RSS capable (which are fairly cheap now), SMB 3 Multi Channel will kick in. SMB 3 Multi Channel establishes multiple TCP channels and engages multiple credits on each TCP channel. So with just 2 TCP channels, you could now have 20MB I/O in flight at any given moment.

In short, if Windows and especially so Windows 2012 is part of your IT environment (or planned environment), make sure your backup target has an upgrade path to SMB 3! And don’t be fooled by just the SMB 3 label! Ask your vendor if it is SMB 3 Multi Channel. The SMB 3 protocol allows a storage device to negotiate SMB 3, but not support SMB 3 Multi Channel!

Wishing you higher backup/restore speeds with SMB 3 Multi Channel!

By dilipcnaik | Also posted in File copy, Hyper-V, SMB 2.2, SMB 3.0, Windows 8, Windows Server 2012 | Comments (0)

The need for Change Block Tracking to perform Differential backups of Hyper-V VMs

November 1, 2013 – 8:21 pm

Hyper-V has been gaining momentum and with Hyper-V 2012 supporting SMB 3.0 based NAS storage, this momentum is likely to accelerate. And of course, any commercial deployment needs to have a proper backup policy. This blog examines some simple, but often overlooked problems in Hyper-V backup, which of course, all backup vendors have tackled in some way or the other.

In general, there are two ways to do a backup with VMs

  1. Backup from within a VM
  2. Backup from the hypervisor aka Hyper-V parent partition

In some particular cases, only of these choices is feasible. Figure 1 shows a VM that uses an iSCSI LUN that is passed through directly to the VM.

HvBack1

After the advent of the VHDX file format and the associated performance and robustness improvements, there are even less reasons to use the configuration depicted in Figure 1. However, in this configuration, the only way to backup is by running a backup application within the VM. The speed boost you get by eliminating the NTFS stack within the parent partition is marginal, and live migration of a VM with this configuration involves a LUN transfer when the iSCSI volume needs to be moved to a different Hyper-V host.

HvBack2

Figure 2 shows a more typical configuration with a VM using a VHD(x) file (VHD or VHDX). Figure 2 shows this VM being backed up from within the VM, even though other choices exist. The main drawback here is that if the Hyper-V host were running 20 such VMs, one would have to pay the cost of 20 Backup App licenses.

HvBack3

Figure 3 shows a VM again using a VHD(x) file, but backup being performed from the Hyper-V parent partition. This is a popular configuration since the cost of the Backup App can be amortized over all the VMs being hosted. The BackUp App depends upon the VSS infrastructure Microsoft has created that runs in both the Hyper-V parent partition and inside the VM. Of course, if the VM is running an OS where no VSS IC requestor exists, this configuration is not feasible.

Once a snapshot is created and a fullback is done, the full backup will include a complete copy of the VHD(x) file. Given that the VHD(x) file will be at least 10s of GBs as in ranging anywhere from 20GB to 100GB or more, it is highly desirable that the subsequent backups be differential backups which only backup changed data within the VHD(x) file.

HvBack4

And that is where the problem lies. None of Windows 2012, Windows 2012 R2, or Hyper-V 2012 provide a facility to determine the changed blocks within the VHD(x) file.

An ideal solution would install in the Hyper-V parent partition and would install, uninstall, load, unload without requiring a reboot of the Hyper-V parent partition. The VMs would have to be restarted for the change tracking to work. This is shown in Figure 5.

HvBack5

Any ISV or OEM looking for such a generic solution is encourage to contact me via LinkedIn.

By dilipcnaik | Also posted in Hyper-V, NTFS, SMB 2.2, SMB 3.0 | Comments (2)

How SMB 3 Witness Protocol detects failure without any timeouts

November 1, 2013 – 2:51 am

The SMB 3 protocol that first shipped with Windows Server 2012 (and Windows 8) is remarkable for making Network Attached Storage (NAS) comparable, and in some senses, even superior to Direct Attached Storage (DAS). NAS is now almost as fast as DAS when used without hardware acceleration. When used with hardware acceleration using the sister protocol SMB Direct also referred to as RDMA, the speed can be even higher! Further, SMB 3.0 based NAS is as reliable since it provides detection of node failures and failover of open file handles (without invalidating the handle), all within a matter of 5 seconds or less. See Jose Barreto blogs for descriptions of SMB Direct and SMB Multi Channel that emphasize the speed aspects of SMB 3.0.

Given that SMB timeouts are of the order of 40 seconds, and TCP timeouts are also of a similar order of time, SMB 3.0 cannot reply upon timeouts to detect failures. This blog explains the basics of how the Witness Protocol works in conjunction with SMB 3.0 to achieve the required failure detection and failover.

This blog provides an overview and is NOT aimed at a developer audience since some technical details are skipped.

It all starts with an SMB 3 client connecting to an SMB 3 clustered file server as shown in Diagram 1

Slide1

The client notices the highly available share and using the Witness Protocol (which is RPC based), requests the node to which it connected for data path access to return a list of IP addresses for each cluster node running the Witness Protocol Service. This is shown in Diagram 2.

Slide2

As shown in diagram 3, the server responds with a list of IP addresses for all cluster nodes running the Witness Service Protocol. The protocol allows for returning both IPv4 and IPv6 addresses.

Slide3

The client receives this information and registers a notification with one of the cluster nodes other than Node A, with which it is already connected to consume data via SMB 3.0. The idea is that the cluster nodes will be running a cluster quorum protocol, whatever it is, and hence the cluster nodes B, C, D (in this example) will notice if and when node A becomes unavailable. This is shown in Diagram 4.

Slide4

Now imagine that node A becomes unavailable for some reason as shown in Diagram 5. The exact reason is immaterial. It could be a power failure or network failure or a system crash or some other reason.

Slide5

Node B (and also C and D) notice that node A is unavailable via the cluster quorum protocol running within the cluster. Node B (in this example) issues an RPC callback to the client notifying it that Node A is unavailable.

Slide6

The client then performs an SMB Session Setup, Tree Connect etc to any one of the other remaining nodes. In Diagram 7 in this example, the client connected to Node C.

Slide7

Note that the “client” can itself be another server e.g. the client could be a SQL server or an IIS server.

By dilipcnaik | Also posted in SMB 2.2, SMB 3.0 | Comments (0)

Tiered Storage and write back caching in Windows Server 2012 R2

July 12, 2013 – 12:13 am

With Windows Server 2012 R2, Microsoft introduces support for tiered storage and write back caching. With only rudimentary details available, this blog examines some highlights and also asks a few questions that I hope to make the content of a future blog.

Tiered storage and write back caching with Windows Server 2012 R2 requires:

  • A Storage Spaces capable set of rotating hard disks i.e. SAS, SATA, or USB hard disks. Obviously USB disks have their limitations in terms of IOPS.
  • A Storage Spaces (set) of flash storage – the word “flash” is used to loosely include SLC, MLC, and other kinds of SSD; again these must be SAS, SATA, or USB
  • Creation of a Storage Space that includes both rotating hard disk and flash disks

Tiered storage in Windows Server 2012 R2 provides just two levels. At any given time, a particular file may be

  • Fully on SSD because that is how Windows decided the file should be
  • Fully on HDD because that is how Windows decided the file should be
  • Partly on SSD and partly on HDD because that is how Windows decided the file should be
  • Pinned fully to either HDD or SSD by the administrator

With tiered storage, Windows tracks access to file ranges with a granularity of 1MB ranges. By default, a scheduled job runs at 1AM and moves the often accessed ranges of the file to SSD and the less often accessed parts of the file to HDD. The retiering can also be run on demand by the system administrator.

Windows Server 2012 R2 also introduces write back caching along with tiered storage. When writes happen, some (or all) of them end up with the new data on SSD tier. Presumably, at a later time when the scheduled optimization job runs, the data is moved to HDD.

The pros of the Windows Server 2012 tiered storage and write back caching:

  • Built into the operating system and free
  • If the understanding that write back caching simply places data on SSD and uses regular file structures is correct, the likelihood of  data corruption due to cache coherency and cache corruption is minimized

The cons of Windows server 2012 tiered storage and write back caching:

  • Only works with Storage spaces which requires SAS, SATA or USB, and in addition requires all storage to be non-RAID
  • Does not work with dedicated SSD designated as cache or in other words, the likelihood of the SSD becoming full and then write back caching being turned off is higher
  • Is not “real time” in the sense that potentially all writes go into the SSD. I could be wrong here since not enough details are available. But certainly, the process of moving often accessed file ranges to SSD, and less often accessed file ranges to HDD is not real time. It does allow the file to still be used during this retiering process, but it is still only periodic and by default only once per day.
  • The retiering process and also the process of monitoring and logging statistics as to what file ranges are actively accessed and what file ranges are less actively accessed may be resource intensive.
By dilipcnaik | Also posted in Hyper-V, NTFS, SMB 2.2, SMB 3.0, SSD, VHDX | Comments (0)

Hyper-V 2012 operations and the importance of SMB 3.0 Multichannel

July 12, 2013 – 12:03 am

Jose Baretto from Microsoft  has put out numerous blogs and talks, including some on SMB 3.0 and Multichannel. Some examples include The basics of SMB 3.0 Multichannel and Windows Server 2012 NIC Teaming and Multichannel. While it is difficult to add to the voluminous material Jose has contributed, this blog highlights the Hyper-V 2012 operational scenarios where SMB 3.0 Multi-Channel is useful and also points out an often overlooked fact of a configuration that is SMB 3.0 Multichannel capable. The goal of the blog is to draw attention to things that Jose has adequately already explained, but people have missed for some reason.

The Hyper-V 2012 team rewrote pieces of Hyper-V to support placing Hyper-V workloads on an SMB 3.0 share. Significantly, Hyper-V uses SMB 3.0 as a transport to move large amount of data/files from one SMB 3.0 NAS to another during live migration. But SMB 3.0 Multichannel is useful in scenarios other than live migration as well.

With Windows Server 2012, Microsoft also rewrote APIs such as the CopyFile API to leverage SMB 3.0 and its performance. CopyFile ensures there are multiple 1MB I/Os in flight and using the SMB 2/3 credit algorithm, the number of I/Os in flight can be increased. Also, these multiple I/Os in flight can flow on parallel TCP connections using SMB 3.0 Multi-Channel. Deploying a VM would typically involve copying a large vhdx file from say the test server to the production server. Another example would be Microsoft System Center copying a large (10s of GBs) vhdx file from a System Center Library Server to the production Hyper-V server. These file copies would benefit from SMB Multichannel.

And that brings me to the last point in this blog. Many folks I have talked to miss the fact that there is a case that allows for Multi-Channel without requiring multiple NICs. Here are the cases where SMB 3.0 Multichannel can come into play:

  • Either client OR server has multiple 1GB NICs. Note that it is not necessary for BOTH client and server to have multiple NICs.
  • Either client OR server has multiple 10GB NICs. Note that it is not necessary for BOTH client and server to have multiple NICs.
  • Both client AND server have a single 10 GB NIC and both the client and server NICs are RSS capable

As Jose Baretto has pointed out multiple times, even when both client and server each have only a single 10GB RSS capable NIC, that is sufficient to enable Multichannel. Microsoft observed that without RSS, TCP send/receive completion interrupts get serviced by a single CPU and that CPU becomes a bottleneck and prevents a typical Windows system from pumping a full 10Gbps through the NIC. A single TCP channel using RSS can alleviate the single CPU bottleneck, but it cannot use the full capability of the 10Gbps NIC. But a Multichannel TCP using RSS does have a chance of saturating the full 10Gbps capability.

Given that backup and vhdx file copy scenarios will occur often in a typical Hyper-V 2012 environment, SMB Multichannel thus not only plays an important role, but is very likely to be enabled with the hardware used for commercial deployment of Hyper-V 2012 VM workloads.

By dilipcnaik | Also posted in File copy, Hyper-V, SMB 3.0, VHDX | Comments (0)

Developing kernel mode drivers for Windows 8

July 17, 2012 – 7:30 pm

I have been developing Windows kernel mode drivers for 10+ years now and notice that the Windows 8/Windows Server 2012 WDK brings some changes. This blog tries to highlight the changes in the hope that other developers will benefit.

I went through the mechanics of installing the Windows 8 WDK on 2 different laptops. So with the caveat “Your mileage may vary – maybe I hit the jackpot and my experience was unique”, here we go:

  1. WDK 8 requires that you first install Visual Studio 2012. See http://msdn.microsoft.com/en-US/windows/hardware/hh852362 and the listed System Requirements section that among other things, state “Before you begin, you must first install Visual Studio Professional 2012 RC or above”
  2. For now, Visual Studio 2012 is “free” since it is not yet a released product. Presumably, two different parts of Microsoft will soon tell us a couple of important data points
    1. The Visual Studio team will tell us pricing for the various different versions/SKUs of Visual Studio 2012
    2. A different part of Microsoft will tell us which SKUs are acceptable for compiling the WDK code samples and code developers write
    3. I am not referring to the new ARM based version of Windows called WindowsRT. I am referring to writing drivers for the x86/X64 platform. Even that one now requires Visual Studio 2012.
    4. Visual Studio is an excellent product for whom it works. A while back – as in 5 or so years ago, I abandoned it, primarily due to the long install time and resources it consumed in terms of disk space. The only use I had for it was the compiler. I use a different editor, and I use WinDbg in stand alone mode.  So when a previous version of the WDK (called the DDK at that time) shipped with compilers, I abandoned Visual Studio. I don’t seem to have the same choice any more.
    5. Depending upon which version of Visual Studio you install, and depending upon what choices you make during the installation, Visual Studio will take some time to install and occupy some GBs (certainly less than 10GB) of disk space.
    6. In case you are still reading, the WDK no longer downloads sample code. My gratitude to the people who posted this fact e.g. http://boardreader.com/thread/Samples_arent_installed_along_with_the_W_u8jjs__3e9c9b67-ea9f-4225-a268-5d5ece555568.html Presumably, this makes it easier for Microsoft to release new or updated code samples without shipping the whole WDK
    7.  The code samples are available at http://code.msdn.microsoft.com/windowshardware . Presumably Microsoft will release some scripts to download the samples in some sort of collection e.g. all the samples, all the USB samples, all the storage driver samples, etc. But meanwhile, one has to download the samples one sample at a time. While this saves on disk space, the savings are miniscule compared to the added GBs occupied by VS 2012.
    8. I must acknowledge that VS 2012 now provides an ARM capable compiler, something the old WDK did not.

We will have to wait and see what the VS 2012 requirement adds in terms of software licensing costs. I guess that is just “the cost of doing business” with Windows 8.

In the meanwhile, I look forward to attending the next plug fest and testing my driver(s) for CSV 2.0 compatibility.

By dilipcnaik | Also posted in NTFS, Windows 8 | Comments (0)

Windows Server 8 NIC Teaming tips

March 12, 2012 – 4:10 pm

Some highly knowledgeable folks at Microsoft recently shared some very valuable tips during the recently concluded MVP Summit. This blog is a small sample of thse tips.

Prior to Windows 8, NIC Teaming has been a feature never officially supported by Microsoft. It was a third party offering from an OEM/IHV/ISV and all support for the feature had to be provided by the third party. I personally have spent considerable time debugging situations where a system start up service I wrote had issues. It turned out that my service could not connect to the Domain Controller because the NIC team was still in the forming stage and had not yet completed its initialization.

Windows Server 8 natively supports NIC teaming. Here are the highlights and tips:

  • NIC teams can only be formed between homogenous NICs. So two 1GB NICs can be teamed, or two 10GB NICs can be teamed, but you cannot team a 1GB and 10GB NIC.
  • If the individual NIC members each support Receive Side Scaling (RSS), the NIC team also supports RSS. Hence it is a good idea to team NICs that support RSS. The resulting NIC team is also highly capable and does not lose any functionality.
  • If the individual NIC members each support RDMA, the resulting NIC team does NOT support RDMA. Given how Windows 8 SMB 2.2 natively supports RDMA without modifying applications, it is a bad idea to team NICs with RDMA capabilities, and where the interconnect (routers, etc) also supports RDMA
By dilipcnaik | Also posted in SMB 2.2, Windows 8 | Comments (1)

Slow file copy or slow file transfer with various Windows versions 2k8, 2k8R2, 2k3

October 7, 2011 – 3:37 am

There are various posts in the Microsoft Technet File Systems and Storage forum (and other forums) indicating slow file copy/transfer speed. The details vary, but most of them involve either Windows Server 2008, Windows Server 2008 R2, or Windows Server 2003. This document is an attempt to coalesce all the various solutions into a single place. Note that the author of this document has not personally tested any of the solutions, but various posts in the forum indicate these to have solved the problem for some users.

Symptom: File copy or File transfer speed is slow, either to a local drive or a network drive, especially so when a Windows Server 2008 or newer Windows file server is involved.

Possible Solutions in no particular Order

  • Make sure both IPv6 and IPv4 are running on 2008 R2, even if the 2008 R2 server is the lone IPv6 device on the network! There may be alternate solutions but this solution has been reported to work [1] [2]
  • Tune TCP on client – set autotuning to off “netsh interface tcp set global autotuninglevel=disabled” [2]
  • Tune TCP on client and server – turn off receive side scaling “netsh interface tcp set global rss=disabled” [3]
  • Tune TCP disable large send offload [5] [17]
  • Tune TCP disable large receive offload [11]
  • Tune TCP – disable  offload [5] [7][9] [11] [16]

Edit the registry key Edit the registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters; create a new DWORD value named DisableTaskOffload and set its value to 1

  • Use non-cached writes while copying the file e.g. run Windows 7 xcopy which supports a new /J switch to do non-cached writes. The Microsoft Performance Team uses the Microsoft Exchange utility eseutil to copy files using non-cached writes. However, some Microsoft customers may not have Microsoft Exchange. Also, the legalities of copying files from a Microsoft Exchange server to a File Server as the Microsoft Performance Team blog in Reference [6] suggests are beyond the scope of this compilation of solutions. [6] [14]
  • Install all hotfixes from Microsoft [8]
  • Don’t use File Explorer to do copy/paste – and if you do, ensure Remote Differential Copy is turned off [4]
  • If one of the drives involved in the file copy is an external USB drive, ensure that you are using the USB 2 protocol and not the older slower USB 1. Start Device Manager, find USB Controller, clock USB2 Enhanced Controller Host, and right click properties; make sure it is enabled. Repeat these steps for all USB2 Enhanced controller hosts.
  • Make sure drive compression is turned off [10]
  • Make sure antivirus is fully updated or if possible, remove antivirus solutions from the equation to see if that is the cause [12]
  • Install all updated drivers from vendors especially NIC vendors and RAID drivers [13] [16]

References:

  1. http://social.technet.microsoft.com/Forums/en-US/winserverfiles/thread/7fd2a924-b1b7-400a-acab-398d09bdb462 What causes Windows 2008 R2 slow file and folder copy
  2. Server 2008 R2 core slow file copy http://social.technet.microsoft.com/Forums/en-US/winservercore/thread/b2462e82-710d-4357-827f-d97172e921f4
  3. Windows 2008 R2 File and Folder Copy very slow over 1000MBs link http://social.technet.microsoft.com/Forums/en-US/windowsserver2008r2general/thread/2e14aa58-cb9b-462c-8a9e-7b10be2de3cb/
  4. Why is Windows 7 still as slow as Vista with File Copy http://answers.microsoft.com/en-us/windows/forum/windows_7-performance/why-is-windows-7-still-as-slow-as-vista-with-file/a5b548ad-9102-40f3-a320-6cd1cabd1f3d?page=3
  5. Slow network file copy on Windows 7 http://serverfault.com/questions/85965/slow-network-file-copy-on-windows-7
  6. Slow large file copy issues  Microsoft Performance team blog http://blogs.technet.com/b/askperf/archive/2007/05/08/slow-large-file-copy-issues.aspx
  7. Slow file copy between Vista and Small Biz Server http://www.smallbizserver.net/Forums/tabid/53/aff/57/aft/62300/afv/topic/Default.aspx 
  8. Low performance when you transfer files from external IEEE 1394 device to server http://support.microsoft.com/kb/975500
  9. Very slow DFSR on Windows Server 2008 http://social.technet.microsoft.com/Forums/en-US/winserverfiles/thread/e55022a4-db65-4dc9-a2f1-96b7f5d8e2fa
  10. Server 2008 File transfer is slow http://windows2008forum.com/topic/339-server-2008-file-share-transfer-is-slow/
  11. File Copy very slow to Windows Server 2008 R2 http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/Server/Windows_Server_2008/Q_25149934.html
  12. IPS driver causes network slowdown http://community.norton.com/t5/Norton-Internet-Security-Norton/Bug-IPS-device-driver-causes-network-slowdown/td-p/151735
  13. Network transfers start fast then slow down http://hardforum.com/showthread.php?t=1566589
  14. Windows 2008 R2 large file copy uses all available memory and slows down http://www.networksteve.com/forum/topic.php/Windows_2008_R2_-_large_file_copy_uses_all_available_memory_and/?TopicId=15382&Posts=5  
  15. Slow LAN transfer Windows Server 2008 R2 http://social.technet.microsoft.com/Forums/en-US/w7itpronetworking/thread/96fe7d15-2bde-451b-8a12-796682a6cea8 – enable IPv6
  16. Windows Server 2008 R2 file/folder copy/paste very slow http://social.technet.microsoft.com/Forums/en-US/windowsserver2008r2general/thread/2e14aa58-cb9b-462c-8a9e-7b10be2de3cb 
  17. Windows 7 Large File copy painfully slow http://social.technet.microsoft.com/Forums/en/w7itpronetworking/thread/c30f6649-a0d9-4f5e-8671-904a8f5469bb 

 

By dilipcnaik | Also posted in File copy | Comments (2)