{"id":500,"date":"2010-10-29T12:47:54","date_gmt":"2010-10-29T19:47:54","guid":{"rendered":"http:\/\/mpsharp.com\/blog\/?p=500"},"modified":"2010-10-29T12:47:54","modified_gmt":"2010-10-29T19:47:54","slug":"jumbo-frames-iscsi-and-disabling-nagle","status":"publish","type":"post","link":"https:\/\/mpsharp.com\/?p=500","title":{"rendered":"Jumbo Frames, iSCSI and Disabling Nagle"},"content":{"rendered":"

So the new project (which I get to spend about 15 minutes a week on) has been to remove the mechanical harddisk from my HTPC<\/a> and have it run completely silently off the SSD<\/a>.\u00a0\u00a0 The first stage was purchasing the ReadyNAS NV+<\/a>.\u00a0 2TB drives have dropped below $100 so a NAS device with 1Gb networking was the no brainer solution.<\/p>\n

Given I have two ATI CableCARD tuners<\/a> (which BTW are no longer being made or supported by ATI), I needed to make sure the network and NAS had enough bandwidth and performance to write two simultaneous HD streams, while reading a third.\u00a0 Now in theory, with a 1Gb network, you should be able to get roughly 100MB thoughput, unfortunately the real world doesn’t work that way.<\/p>\n

My initial testing using Lan Speed Test<\/a> (more on that later) showed I was getting around 24MBps writes and 45MBps reads.\u00a0 (Remember big Bs are Bytes and little bs are bits.\u00a0 8 bits to a byte)\u00a0 So looking at my Comcast HD recordings, it turns out Fox is broadcasting 1920×1080 MPEG2 at around 14Mbps. \u00a0 This isn’t that bad.\u00a0 Note that DirectTV uses MPEG4 at around 5Mbps.\u00a0 Since mp4 delivers better quality at lower bit rates,\u00a0 the DirectTV advertisements aren’t lying when they say the picture is better.\u00a0\u00a0 For comparison, a typical DVD is MPEG2 @ 9.8Mbs and a BlueRay is MPEG4 at 40Mbps.\u00a0\u00a0 So if you haven’t figured it out yet, HD can really mean anything you want.<\/p>\n

But the answer is if you want to record 2 HD TV channels from Comcast, you need about 2MBps per stream, or really just 4MBs, but you definitely want to have some head room, and there’s also the need to play DVD images back from the drive<\/p>\n

I was hoping to get a little better than 25% of maximum throughput so started looking for solutions.\u00a0\u00a0\u00a0 The first thing I poked at was the Nagle’s algorithm<\/a> attributes.\u00a0\u00a0\u00a0 This basically tells the TCP stack to gather all the small requests in to one big one before sending.\u00a0\u00a0\u00a0\u00a0 Turns out gamers want to disable this feature to make sure every keystroke\/click is sent the moment they enter it, and not let things batch up on the network card.\u00a0\u00a0\u00a0 For media streaming, you want the opposite behavior, but you’re rarely sending small data packets anyways. \u00a0\u00a0 But just for kicks, I set the following registry settings<\/p>\n

HKLM\/SYSTEM\/CurrentControlSet\/services\/Tcpip\/Parameters\/Interfaces\/{nicid}\/<\/span><\/p>\n

GlobalMaxTcpWindowSize = 0x01400000 (DWORD)
\nTcpWindowSize = 0x01400000 (DWORD)
\nTcp1323Opts = 3 (DWORD)
\nSackOpts = 1 (DWORD)
\nTcpAckFrequency = 4 (DWORD)
\nTcpDelAckTicks = 2 (DWORD)<\/span><\/p>\n

And ran this command:<\/p>\n

C:\\ netsh interface tcp set global rss=disabled chimney=disabled autotuninglevel=disabled congestionprovider=none<\/span><\/p>\n

Mostly from recommendations from this iSCSI site<\/a> (more on that below):<\/p>\n

Basically I’m saying the opposite.\u00a0\u00a0 Package up as many small bits as possible into larger ones to avoid the overhead.\u00a0 Difficult to measure the differences here, but I’m recording them here for myself in case I run into problems.\u00a0\u00a0 Though, it turns out there is another TCP feature along the same lines that does help with streaming media and was much more noticeable.<\/p>\n

I had already done all the cache optimizations possible on the ReadyNAS, and configured it for Raid 0 since I really don’t need redundancy for recorded TV shows and I’m using Amazon S3 for offsite backup as described here<\/a>.\u00a0\u00a0 One of the features I found on the ReadyNAS was support for TCP Jumbo Frames<\/a>.\u00a0\u00a0 So it turns out the standards the Internet still runs on today were defined over 30 years ago.\u00a0\u00a0 Given the reliability of Ethernet at the time, the designers decided that 1500 bytes was the largest amount of data to be communicated in each packet so if the receiver didn’t get the packet, the resend wouldn’t be so large.\u00a0\u00a0\u00a0 In today’s home gigabit switched networks, collisions and data corruption are almost unheard of.\u00a0 So rather than waste all the CPU & interrupt time splitting and joining small packets, you just build one big one.\u00a0\u00a0 This is also a bit more efficient because each packet also requires header and footer\u00a0 information describing where it should be delivered to.\u00a0\u00a0 Unfortunately, because everyone has to follow a standard, the largest the Jumbo Frame packet goes to is 9K, but that’s still almost a 4x increase in the data delivered with the same header and footer used for the original frame size.<\/p>\n

So I started looking at the configuration for the on-board network adapter on my Intel P35<\/a>.\u00a0 No Jumbo Frame option, but this I found this note:<\/p>\n

Note: The Intel PRO\/1000 PL Network Connection supports jumbo frames in Microsoft* Windows* operating systems only when Intel\u00ae PROSet for Windows Device Manager<\/a> is installed.<\/p>\n

Cool!\u00a0 So Installed Intel ProWin and still couldn’t find the JF option.\u00a0\u00a0 Do a little more research and find this:<\/p>\n

The following gigabit LAN components included with Intel\u00ae Desktop Boards do not support jumbo frames:<\/p>\n

\n