Jose Baretto from Microsoft has put out numerous blogs and talks, including some on SMB 3.0 and Multichannel. Some examples include The basics of SMB 3.0 Multichannel and Windows Server 2012 NIC Teaming and Multichannel. While it is difficult to add to the voluminous material Jose has contributed, this blog highlights the Hyper-V 2012 operational scenarios where SMB 3.0 Multi-Channel is useful and also points out an often overlooked fact of a configuration that is SMB 3.0 Multichannel capable. The goal of the blog is to draw attention to things that Jose has adequately already explained, but people have missed for some reason.
The Hyper-V 2012 team rewrote pieces of Hyper-V to support placing Hyper-V workloads on an SMB 3.0 share. Significantly, Hyper-V uses SMB 3.0 as a transport to move large amount of data/files from one SMB 3.0 NAS to another during live migration. But SMB 3.0 Multichannel is useful in scenarios other than live migration as well.
With Windows Server 2012, Microsoft also rewrote APIs such as the CopyFile API to leverage SMB 3.0 and its performance. CopyFile ensures there are multiple 1MB I/Os in flight and using the SMB 2/3 credit algorithm, the number of I/Os in flight can be increased. Also, these multiple I/Os in flight can flow on parallel TCP connections using SMB 3.0 Multi-Channel. Deploying a VM would typically involve copying a large vhdx file from say the test server to the production server. Another example would be Microsoft System Center copying a large (10s of GBs) vhdx file from a System Center Library Server to the production Hyper-V server. These file copies would benefit from SMB Multichannel.
And that brings me to the last point in this blog. Many folks I have talked to miss the fact that there is a case that allows for Multi-Channel without requiring multiple NICs. Here are the cases where SMB 3.0 Multichannel can come into play:
- Either client OR server has multiple 1GB NICs. Note that it is not necessary for BOTH client and server to have multiple NICs.
- Either client OR server has multiple 10GB NICs. Note that it is not necessary for BOTH client and server to have multiple NICs.
- Both client AND server have a single 10 GB NIC and both the client and server NICs are RSS capable
As Jose Baretto has pointed out multiple times, even when both client and server each have only a single 10GB RSS capable NIC, that is sufficient to enable Multichannel. Microsoft observed that without RSS, TCP send/receive completion interrupts get serviced by a single CPU and that CPU becomes a bottleneck and prevents a typical Windows system from pumping a full 10Gbps through the NIC. A single TCP channel using RSS can alleviate the single CPU bottleneck, but it cannot use the full capability of the 10Gbps NIC. But a Multichannel TCP using RSS does have a chance of saturating the full 10Gbps capability.
Given that backup and vhdx file copy scenarios will occur often in a typical Hyper-V 2012 environment, SMB Multichannel thus not only plays an important role, but is very likely to be enabled with the hardware used for commercial deployment of Hyper-V 2012 VM workloads.
winntfs 