Thursday, May 31, 2012

10 GbE performance troubleshooting 1

On a Windows 2003 server with 1 GbE NIC and DSN writing to DataDomain device, I can see about 140 MB/s (dedup happens on the DSN and NIC utilization is approx 15%).

Now, with 2 Windows 2008 R2 servers setup with 10GbE, I copy a file from one Windows 2008 R2 to another one.  It at most utilizes 12% of the 10GbE.  If I add write another file at the same time, I see the utilizes 20% of the 10 GbE.  I follow some of the suggestions by Cisco to tweat the OS (only thing I have not done is Jumbo frame).  However, I don't see much improvement.

After doing more research, it looks like it is an OS limitation.  See kb article from HP site.

"There was still perceived TCP performance issue, but it turned out to be a matter of limitations in performance per thread in Windows Server 2003. For instance, if copying only one file from one server hosting a NC522SFP to another using a NC522SFP, only a small fraction of the theoretical 10-Gigabit performance was achieved. However, if multiple sessions were run simultaneously, similar performance gains were seen as with UDP. In other words, the bottleneck was not the NIC."

Hopefully, I will have more time to run test and determine the limitation in the summer.  Not sure if Linux / Unix will do a better job.

No comments: