In comp.protocols.tcp-ip jbrockerville wrote:
> My setup is as follows, I have my server app sitting on a WINXP
> laptop on the network and my client app on my WINXP workstation. I
> connect and can transfer the desired data in about 43 ms. Here is
> the odd part, I can open any of the above mention apps (while my app
> is still connected no less) and can now transfer the same data in
> about 7 ms - a six-fold increase! And ofcourse, when I close the
> app my transfer time reverts back to about 43 ms.

Perhaps your application is not interacting well with whatever
interrupt mitigation is being employed and some of your packets get
"stuck" on the NIC waiting for a timer (on the NIC) to expire to
trigger the interrupt that tells the host the packets are there.
Increase the traffic rage by starting one of the other apps and you
might get enough packets to have the interrupts fire sooner.

Not the same OS, but the principles should apply:

ftp://ftp.cup.hp.com/dist/networking...cy_vs_tput.txt

rick jones
--
denial, anger, bargaining, depression, acceptance, rebirth...
where do you want to be today?
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

jbrockerville wrote:

> My setup is as follows, I have my server app sitting on a WINXP laptop
> on the network and my client app on my WINXP workstation. I connect
> and can transfer the desired data in about 43 ms. Here is the odd
> part, I can open any of the above mention apps (while my app is still
> connected no less) and can now transfer the same data in about 7 ms - a
> six-fold increase! And ofcourse, when I close the app my transfer time
> reverts back to about 43 ms.

Makes perfect sense. If you were the only one sending packages to
Hawaii, do you think FedEx would send a plane there every night? You
are benefiting from the more frequent execution of code in various
parts of the network stack.

Consider, for example, a typical input interrupt colescer. When a
single packet is received, it sets a timer to trigger an interrupt in 2
milliseconds or when the receive buffer is at least half full. No other
traffic, the timer fires in 2 milliseconds. Lots of other traffic, the
timer fires when the receive buffer is half full.

DS

David Schwartz wrote:

> Makes perfect sense. If you were the only one sending packages to
> Hawaii, do you think FedEx would send a plane there every night? You
> are benefiting from the more frequent execution of code in various
> parts of the network stack.
>
> Consider, for example, a typical input interrupt colescer. When a
> single packet is received, it sets a timer to trigger an interrupt in 2
> milliseconds or when the receive buffer is at least half full. No other
> traffic, the timer fires in 2 milliseconds. Lots of other traffic, the
> timer fires when the receive buffer is half full.
>
> DS

Thanks the for replies. I do agree with your analogy however, if I
were to begin downloading a large file from some server and run my test
while downloading I do not see the same increase in speed. Correct me
if I'm wrong, but downloading a large file while running my test should
be meeting the above 2 posts requirements that my rx buffer be filling
faster than my test is filling it and therefore be triggering at faster
rate.

Also, I tried having a look at the rx bytes and packets using the
performance monitor in admin tools and found that while the afore
mentioned streaming apps are running, there is no noticable increase in
network traffic. Are there any other "finer grain" tools out there
that I can use? Tools that would tell me where data is being directed?

I read up on the winsock QOS stuff as well. I'd have to re-write some
code to attempt that. A good suggestion, but I'd rather avoid changing
up some very stable code at this time. For now, I hope someone can
suggest a clearer cause and possible solution.

Like I said before, completely stumped.

jb

"jbrockerville" wrote in message
news:1156284960.609583.98240@74g2000cwt.googlegrou ps.com...
> Like I said before, completely stumped.

Me too. However, I will suggest that if your program is not able to
maximize its use of the available network bandwidth, there is likely at
least some kind of problem with the efficiency of your own code.

It may well be that the other applications somehow "interfere" in a way that
reduces or eliminates that inefficiency, but it's not normal for a
well-written application to not be able to use nearly 100% of the network
speed available, even on a fast LAN.

If you think that your code is already efficient, then you should be looking
for something broken on the PC where you run the applications that speed
things up (you don't actually specify on which PC running them helps). This
includes a fresh install of the operating system (into a different partition
as a dual-boot setup, if that's easier for you) if nothing else turns up.
If you somehow manage to ensure everything is copasetic on that PC, then try
taking the router out of the picture (use a cross-over cable to connect the
computers directly). If after all of that, you are still seeing a slowdown,
I think you are back to inefficient code on your part, no matter how good it
looks to you.

I have seen trivial Winsock implementations do just fine using up the
available bandwidth for peer-to-peer stuff (such as it sounds you've
implemented), without doing any fancy optimizations. So the behavior you're
seeing is not normal for a decently-written program.

Pete

May I ask you what you mean when you wrote about 43/7 ms, if that measument
of returning from send() or time from send() and return from recv() on the
peer one ?
TIA
Arkady

"jbrockerville" wrote in message
news:1156264154.976049.167540@74g2000cwt.googlegro ups.com...
> For a sockets newbie, I'm completely stumped by the behaviour that my
> client/server app is showing while other streaming apps are open (i.e.
> Windows Media Player, MSN Messenger, Windows Live Mail, Quicktime
> player, etc.).
>
> My setup is as follows, I have my server app sitting on a WINXP laptop
> on the network and my client app on my WINXP workstation. I connect
> and can transfer the desired data in about 43 ms. Here is the odd
> part, I can open any of the above mention apps (while my app is still
> connected no less) and can now transfer the same data in about 7 ms - a
> six-fold increase! And ofcourse, when I close the app my transfer time
> reverts back to about 43 ms.
>
> I have searched the forums and example projects available on the web,
> tried optimizing WINXP settings (which require a reboot, so rule them
> out I guess). Nothing has given me any insight as to how these apps
> change settings to somehow affect my apps socket.
>
> Does anyone know what the heck is going on here? Can I set some
> options using ioctlsocket() or setsockopt()? Is there actually some
> fundemental difference between my basic send-and-receive-type
> connection and a streaming connection?
>
> Thanks in advance!
>
> jb
>

Again, a good suggestion. I was pretty close to stripping everything
down to a basic, single threaded app to test this theory. Maybe I'll
try that today. I've suspected that the implemented architecture that
I have inherited could be flawed.

As for those timings, some understandable confusion there I guess.
What I meant was 43 ms from the time the client requests data from the
server until the client fully receives it, i.e. a round trip.

jb

wrote in message
news:1156352666.826069.36290@h48g2000cwc.googlegro ups.com...
> [...]
> As for those timings, some understandable confusion there I guess.
> What I meant was 43 ms from the time the client requests data from the
> server until the client fully receives it, i.e. a round trip.

Well, that's a LOT easier to explain, at least in the sense that you're not
necessarily seeing a throughput issue, but rather a ping issue. That is,
once you start sending, a well-written application should be able to keep
the network "pipe" filled and accomplish maximum performance given the
available bandwidth. But even the best-written application has little
control over how long an atomic, solitary transmission takes to get to its
destination, and how long the reply (that is, the other end ramping up and
starting to send the data back, as well as the initial data transmission
beginning) takes.

As a more concrete example to what David was talking about, if you send a
small amount of data, Winsock (or rather, TCP/IP) may not transmit it
immediately (Nagle). This could cause a delay when you are the only program
using the network. If there is other network traffic, it's possible that
your transmission can "hitch a ride", so to speak, on existing traffic and
not have to wait for the Nagle algorithm to time out.

I still don't have anything useful to offer as to why the other programs
improve your performance there without actually using the network. How they
may interact with your program to change the behavior, I can't say. I'll
still point out that you still haven't told us whether it's running the
other programs on the server or the client PC that helps things. That could
be an important component of figuring out what's going on. It's also
possible that there actually is network traffic you're not noticing, and
that you do need to use more reliable tools to identify that fact. Those
tools may even give you insight as to the exact timing of the data you're
trying to send and whether or not it gets combined with other traffic at any
point.

Pete

Could be as simple as somebody else updating the ARP cache...

--
=====================================
Alexander Nickolov
Microsoft MVP [VC], MCSD
email: agnickolov@mvps.org
MVP VC FAQ: http://www.mvps.org/vcfaq
=====================================

"Peter Duniho" wrote in message
news:12ep4eh7m62a3e5@corp.supernews.com...
> wrote in message
> news:1156352666.826069.36290@h48g2000cwc.googlegro ups.com...
>> [...]
>> As for those timings, some understandable confusion there I guess.
>> What I meant was 43 ms from the time the client requests data from the
>> server until the client fully receives it, i.e. a round trip.
>
> Well, that's a LOT easier to explain, at least in the sense that you're
> not necessarily seeing a throughput issue, but rather a ping issue. That
> is, once you start sending, a well-written application should be able to
> keep the network "pipe" filled and accomplish maximum performance given
> the available bandwidth. But even the best-written application has little
> control over how long an atomic, solitary transmission takes to get to its
> destination, and how long the reply (that is, the other end ramping up and
> starting to send the data back, as well as the initial data transmission
> beginning) takes.
>
> As a more concrete example to what David was talking about, if you send a
> small amount of data, Winsock (or rather, TCP/IP) may not transmit it
> immediately (Nagle). This could cause a delay when you are the only
> program using the network. If there is other network traffic, it's
> possible that your transmission can "hitch a ride", so to speak, on
> existing traffic and not have to wait for the Nagle algorithm to time out.
>
> I still don't have anything useful to offer as to why the other programs
> improve your performance there without actually using the network. How
> they may interact with your program to change the behavior, I can't say.
> I'll still point out that you still haven't told us whether it's running
> the other programs on the server or the client PC that helps things. That
> could be an important component of figuring out what's going on. It's
> also possible that there actually is network traffic you're not noticing,
> and that you do need to use more reliable tools to identify that fact.
> Those tools may even give you insight as to the exact timing of the data
> you're trying to send and whether or not it gets combined with other
> traffic at any point.
>
> Pete
>

In comp.protocols.tcp-ip Peter Duniho wrote:
> As a more concrete example to what David was talking about, if you
> send a small amount of data, Winsock (or rather, TCP/IP) may not
> transmit it immediately (Nagle). This could cause a delay when you
> are the only program using the network. If there is other network
> traffic, it's possible that your transmission can "hitch a ride", so
> to speak, on existing traffic and not have to wait for the Nagle
> algorithm to time out.

I'm afraid that is not how Nagle works. The scope of the Nagle
algorithm is the single connection. Multiple connections do not come
into play. So, running another application would have no bearing on
the behaviour of the Nagle algorithm on another application's
connections.

Where things can come into play would be with interrupt coalescing on
the NIC(s). That does though assume those other applications are doing
_something_ over the NIC(s) as well:

ftp://ftp.cup.hp.com/dist/networking...cy_vs_tput.txt

rick jones

for descriptions of how Nagle works (or rather is supposed to work)
there are the works of W Richard Stevens and also previous posts on
the topic one could find with a websearch. in particular, a small send
on an otherwise idle connection should _never_ be delayed by Nagle.

--
The glass is neither half-empty nor half-full. The glass has a leak.
The real question is "Can it be patched?"
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

In comp.protocols.tcp-ip Alexander Nickolov wrote:
> Could be as simple as somebody else updating the ARP cache...

That should be a one-time thing though and a continuous one right?
One would see occasional spikes in response time if something were
held-up behind an ARP resolution, but not a sustained difference.

rick jones
--
oxymoron n, commuter in a gas-guzzling luxury SUV with an American flag
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

jbrockerville wrote:
>For a sockets newbie, I'm completely stumped by the behaviour that my
>client/server app is showing while other streaming apps are open (i.e.
>Windows Media Player, MSN Messenger, Windows Live Mail, Quicktime
>player, etc.).
[...]
>My setup is as follows, I have my server app sitting on a WINXP laptop
>on the network and my client app on my WINXP workstation. I connect
>and can transfer the desired data in about 43 ms. Here is the odd
>part, I can open any of the above mention apps (while my app is still
>connected no less) and can now transfer the same data in about 7 ms - a
>six-fold increase! And ofcourse, when I close the app my transfer time
>reverts back to about 43 ms.
[...]
>Does anyone know what the heck is going on here? Can I set some
>options using ioctlsocket() or setsockopt()? Is there actually some
>fundemental difference between my basic send-and-receive-type
>connection and a streaming connection?


It MAY be that the clue here is that this is triggered by multimedia
applications, not the streaming part.

Many multimedia applications needs precise timing and uses high
resolution timers for this. My understanding is that under SOME
circumstances this can have subtle but system wide effects in Windows
XP/2003, that it may reschedule processes more often and/or
differently when there's at least one active HR timer.

If this is involved I wouldn't be surprised if the details here also
depends on what HAL you're using on the machine (the 4 main ones being
single CPU/multi CPU, classic/ACPI), which hardware it has (and that
the BIOS has enabled!) and details in all other involved hardware (NIC
hardware & drivers).

IIRC I've seen some people claiming to have pinpointed signficant
difference in TCP performance depending on whether a multimedia
application was running or not, the Usenet discussion ended with this
as the best guess.

I've certainly seen some differences in behavior in some applications
when certain other programs is running. The main example is that
SOMETIMES iTunes starts glitching (broken up sound) when starting up
MPC or WMP (never seen it with any non-multimedia application).
Pausing and resuming sometimes fixes it, other times it won't get back
to normal until MPC/WMP has exited... Whether this is caused by this
or not is an open question though.

But if you want more help you'll probably need to provide more details
such as protocol (TCP/UDP), network layout, server OS, nagle on/off
and possibly some sample code...

Peter Duniho wrote:

> As a more concrete example to what David was talking about, if you send a
> small amount of data, Winsock (or rather, TCP/IP) may not transmit it
> immediately (Nagle). This could cause a delay when you are the only program
> using the network. If there is other network traffic, it's possible that
> your transmission can "hitch a ride", so to speak, on existing traffic and
> not have to wait for the Nagle algorithm to time out.

This is only supposed to happen with data on the same TCP connection.
So it should not affect his case. (It's not clear whether you intended
it to suggest that it could or just as another example of how one thing
can speed up another.)

DS

"David Schwartz" wrote in message
news:1156372751.553283.36230@b28g2000cwb.googlegro ups.com...
> This is only supposed to happen with data on the same TCP connection.
> So it should not affect his case. (It's not clear whether you intended
> it to suggest that it could or just as another example of how one thing
> can speed up another.)

You're right, of course. I simply mentioned Nagle as a common example of
the general idea that would be relatively well-known (versus the lower-level
network adapter sorts of things that, at least here in the Winsock
newsgroup, are not necessarily as well known).

But even so, that only explains things if there is indeed other traffic on
the network adapter. If the other applications are indeed just sitting
there doing nothing, it's hard to imagine how they would affect the
performance of the program in question at all.

Seems like if the OP really wants an answer, then the first step is some
genuine inspection of the actual network traffic to see if there's anything
in there that suggests an answer. As well as to eliminate other potential
causes (as I mentioned in a different post).

Pete

Torbjorn Lindgren wrote:
> It MAY be that the clue here is that this is triggered by multimedia
> applications, not the streaming part.
>
> Many multimedia applications needs precise timing and uses high
> resolution timers for this. My understanding is that under SOME
> circumstances this can have subtle but system wide effects in Windows
> XP/2003, that it may reschedule processes more often and/or
> differently when there's at least one active HR timer.


What happens is an application can increase the effective resolution of
the clock tick. Normally the default is 10ms, but many MM apps set it
to 1ms. This can cause code like:

for (i=0; i<1000; i++)
Sleep(1);

to run considerably faster. Without a MM app running, this will take
about 10 seconds, and about 1 second with an MM app running. If the
application has things like timeout parameters on Wait* or Select* type
operations the same applies.

Now this is not usually an issue problem if the application is just
sending and receiving data and blocking on the socket, or blocking on
some sort of synchronization object. But you often see code like:

for(;
{
recv(); //non-blocking
if (got something)
{...}
Sleep(1);
}

Anyway, the OP should try adding a:

timeBeginPeriod(1);

At the top of their program (and a timeEndPeriod(1); at the end).

Peter Duniho wrote:

> Seems like if the OP really wants an answer, then the first step is some
> genuine inspection of the actual network traffic to see if there's anything
> in there that suggests an answer. As well as to eliminate other potential
> causes (as I mentioned in a different post).

I think the most likely explanation is that there's some bug or
questionable logic in his code and the other system activity mitigates
the harm that bug causes.

For example, an idle system might typically reschedule on the next
clock tick (10 mS, say) when it gets a 'Sleep(1)', but an active system
might reschedule the thread as soon as a thread from another process
completes its job (which may be much less than that). The fix is to
figure out why it's calling 'Sleep(1)' and do something more
appropriate (which is most likely nothing).

DS

Thanks for everyone that replied. I tried a lot of different things
and learned some new stuff along the way. The cause, it seems, was a
Sleep(1) in a thread waiting for server data to be put in a queue from
a socket listening thread. This has always been there and I never saw
the speeding up thing until recently since I almost always had one of
those streaming apps open. It was one of those things that I just
caught and was freakishly annoyed by.

Anyways, I commented it out and my round trip time went down to about
3ms and achieves this speed under many different configs. In fact, my
time slows down now when I have one of those other apps open! :P

That said, I orginally put the sleep in there to prevent unwarrented
CPU useage because the data needs processing once returned and with the
CPU maxed out there will be issues with that as this whole process is
time critical. So, I now pose a different question, but related to
this socket business none the less. Given that data could be sent to
my client app at anytime, unlike a straight forward
request-data-recieve-data senario, what would be the best way to
prevent the crazy CPU usage I'm seeing due to one thread waiting for
data from another thread as described above and still achieve the
maximum network speed that my client app has now acheived? Make sense?

Cheers,
jb

j_brockerville@hotmail.com wrote:

> That said, I orginally put the sleep in there to prevent unwarrented
> CPU useage because the data needs processing once returned and with the
> CPU maxed out there will be issues with that as this whole process is
> time critical. So, I now pose a different question, but related to
> this socket business none the less. Given that data could be sent to
> my client app at anytime, unlike a straight forward
> request-data-recieve-data senario, what would be the best way to
> prevent the crazy CPU usage I'm seeing due to one thread waiting for
> data from another thread as described above and still achieve the
> maximum network speed that my client app has now acheived? Make sense?

Fix the bug that causes the crazy CPU usage. Without seeing the code,
it's hard to know what the bug is. It may be code like this:

lock_something();
while(!data_avail)
{
unlock_something();
Sleep(1);
lock_something();
}

That code is just horribly broken. One fix is to use an event. When you
take the last byte of data, block the event while you hold the lock.
When you put data on an empty queue, unlblock the event while you hold
the lock. Then replace the 'Sleep(1)' with a block on the event.

DS