TCP Stream in Win XP Pro? - Networking

Within a GigE network infrastucture, I've got an XP SP2 Pro system, all
updates installed, NIC is Intel Pro 1000 MT Server Adapter, with most
recent driver from intel. I'm testing TCP-IP streaming from that client
to my linux server which has two Pro 1000 MT server adapters in a bond.
I run netperf-2.1pl1.exe from that XP client to a linux server running
netserver, and I can't get more than 30-35MB/sec. Yet my linux clients,
with similar hardware, get 90-100MB/sec netperf to the same exact linux
server. It's not a load issue, as I've run the tests when the network is
idle or busy, same results.

Every single Windows XP client in our domain behaves the same way, just
can't break the 35MB/sec TCP stream rate. Yet linux can break it easy no
problem, so I doubt the problem is in the networking infrastructure.
Problem ius in Windows XP.

Any idea what's going on here? Does anyone else get TCP-IP stream rates
greater than 35MB/sec on windows with the Intel Pro 1000 MT server
adapter? Any OS tunning I need to do? I'd like to see at least
80-90MB/sec, similar to what linux can see.

More than willing to give more detail if needed.

Thanks!

Alex

Alex Lazarevich wrote:
> Within a GigE network infrastucture, I've got an XP SP2 Pro system,
> all updates installed, NIC is Intel Pro 1000 MT Server Adapter, with
> most recent driver from intel. I'm testing TCP-IP streaming from
> that client to my linux server which has two Pro 1000 MT server
> adapters in a bond. I run netperf-2.1pl1.exe from that XP client to
> a linux server running netserver, and I can't get more than
> 30-35MB/sec. Yet my linux clients, with similar hardware, get
> 90-100MB/sec netperf to the same exact linux server. It's not a load
> issue, as I've run the tests when the network is idle or busy, same
> results.

> Every single Windows XP client in our domain behaves the same way,
> just can't break the 35MB/sec TCP stream rate. Yet linux can break
> it easy no problem, so I doubt the problem is in the networking
> infrastructure. Problem ius in Windows XP.

> Any idea what's going on here? Does anyone else get TCP-IP stream
> rates greater than 35MB/sec on windows with the Intel Pro 1000 MT
> server adapter? Any OS tunning I need to do? I'd like to see at
> least 80-90MB/sec, similar to what linux can see.

What is the CPU utilization of each CPU in the system(s)? Which
specific linux kernels are you using? Are they supporting TSO?
(ethtool -k IIRC)

What sort of PCI bus is present in the systems? PCI-4X (64 bit, 66
MHz) or better still PCI-X?

What is the actual TCP window size on the wire? (packet trace of the
SYN's) IIRC there may be some small disconnect between the -S and -s
parameters of netperf and the actual window sizes used by Windows.

Generally speaking, to get good netperf numbers over Gigabit
neworking, one wants a window size of 64KB or better. There are of
course exceptions.

You might also want to test single-byte TCP_RR performance. Related
to the window size above, the single-byte TCP_RR test will be a decent
first approximation of the RTT between the two systems:

Tput <= Window/RTT

Might want to see if Windows is using large send.

rick jones
--
a wide gulf separates "what if" from "if only"
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to raj in cup.hp.com but NOT BOTH...

Rick Jones wrote:

> Alex Lazarevich wrote:
>
>>Within a GigE network infrastucture, I've got an XP SP2 Pro system,
>>all updates installed, NIC is Intel Pro 1000 MT Server Adapter, with
>>most recent driver from intel. I'm testing TCP-IP streaming from
>>that client to my linux server which has two Pro 1000 MT server
>>adapters in a bond. I run netperf-2.1pl1.exe from that XP client to
>>a linux server running netserver, and I can't get more than
>>30-35MB/sec. Yet my linux clients, with similar hardware, get
>>90-100MB/sec netperf to the same exact linux server. It's not a load
>>issue, as I've run the tests when the network is idle or busy, same
>>results.
>
>
>>Every single Windows XP client in our domain behaves the same way,
>>just can't break the 35MB/sec TCP stream rate. Yet linux can break
>>it easy no problem, so I doubt the problem is in the networking
>>infrastructure. Problem ius in Windows XP.
>
>
>>Any idea what's going on here? Does anyone else get TCP-IP stream
>>rates greater than 35MB/sec on windows with the Intel Pro 1000 MT
>>server adapter? Any OS tunning I need to do? I'd like to see at
>>least 80-90MB/sec, similar to what linux can see.
>
>
> What is the CPU utilization of each CPU in the system(s)? Which
> specific linux kernels are you using? Are they supporting TSO?
> (ethtool -k IIRC)
>
> What sort of PCI bus is present in the systems? PCI-4X (64 bit, 66
> MHz) or better still PCI-X?
>
> What is the actual TCP window size on the wire? (packet trace of the
> SYN's) IIRC there may be some small disconnect between the -S and -s
> parameters of netperf and the actual window sizes used by Windows.
>
> Generally speaking, to get good netperf numbers over Gigabit
> neworking, one wants a window size of 64KB or better. There are of
> course exceptions.
>
> You might also want to test single-byte TCP_RR performance. Related
> to the window size above, the single-byte TCP_RR test will be a decent
> first approximation of the RTT between the two systems:
>
> Tput <= Window/RTT
>
> Might want to see if Windows is using large send.

Thanks for the informative reply Rick, I think it's the socket size that
is the problem. I noticed the netperf send/receive socket size on linux
<-> linux was 16384, whereas the windows clients are only 8192. So I'm
trying to run netperf on windows, resetting the socket size to 16384
with this command:

netperf -a16384 -Hnetserver.host.name -fM

But netperf still runs with only 8192 socket size. I also tried -o16384
and -W16384 but still, I can't get the send socket size to 16384. help
pages are terrible.

The question then remains how do I force windows XP Pro to use the
larger socket size on all TCP transmitions? Is there a flag to the
boot.ini?

Thanks for your help,

Alex

Alex Lazarevich wrote:
> Thanks for the informative reply Rick, I think it's the socket size that
> is the problem. I noticed the netperf send/receive socket size on linux
> <-> linux was 16384, whereas the windows clients are only 8192. So I'm
> trying to run netperf on windows, resetting the socket size to 16384
> with this command:

> netperf -a16384 -Hnetserver.host.name -fM

> But netperf still runs with only 8192 socket size. I also tried -o16384
> and -W16384 but still, I can't get the send socket size to 16384. help
> pages are terrible.

Netperf 2.1 is rather old and the usage strings were probably pretty
weak. 2.3 is out there (not sure about windows binary availability)
and I'm working on the "configure-script-enabled" 2.4.0 at the moment.

As you have already surmised, none of those options are the right ones
for netperf. To set the socket buffer size(s) one uses the
_test-specific_ -s and -S options:

netperf -H remote -fM -- -s 16384 -S 16384

The -a option controls local send buffer _alignment_ not send socket
buffer size. The -o option controlled the _offset_ into the buffer
from which data was sent, and -W has controlled the "width" of the
send and receive buffer rings.

By default, netperf uses a "ring" of buffers for send or recv calls
that is equal to the socket buffer size, divided by the send size,
plus one. This can sometimes result in differences in reported
performance when compared to other benchmarks that use a single buffer
for send and/or recv - and thus may fit a bit better in lower
(higher?) levels of the processor data cache hierarcy.

> The question then remains how do I force windows XP Pro to use the
> larger socket size on all TCP transmitions? Is there a flag to the
> boot.ini?

No clue on that one. I've always ass-u-me-d that it invovlved the
registry. Just a wag, but searching for "default tcp window" on
microsoft.com may find something useful.

happy benchmarking,

rick jones
--
firebug n, the idiot who tosses a lit cigarette out his car window
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to raj in cup.hp.com but NOT BOTH...