Fast inter process communication - OS2

This is a discussion on Fast inter process communication - OS2 ; Currently using named shared memory and Dos32(Wait|Post)EvenSem to filter file access. The purpose of the communication is to provide cache and prefetch functions. The current implementation is slow even if it would have 100% cache hit ratio, because there seem ...

+ Reply to Thread
Results 1 to 6 of 6

Thread: Fast inter process communication

  1. Fast inter process communication

    Currently using named shared memory and
    Dos32(Wait|Post)EvenSem to filter file access.

    The purpose of the communication is to provide
    cache and prefetch functions.
    The current implementation is slow even if it would have
    100% cache hit ratio, because there seem to
    be circa 4ms overhead for every access,
    making it slower 2 times compared my average
    harddisk access.

    Part of the overhead is buffer copy to shared
    memory, maybe the current copy implementation
    to/from shared memory could improved, but
    major slowdown still seems to be the process
    switch.

    Any idea for faster communication?


    library functions that use
    --
    Veit Kannegieser

  2. Re: Fast inter process communication

    On Sat, 11 Dec 2004 13:54:48 UTC, "Veit Kannegieser"
    wrote:

    > Currently using named shared memory and
    > Dos32(Wait|Post)EvenSem to filter file access.
    >
    > The purpose of the communication is to provide
    > cache and prefetch functions.
    > The current implementation is slow even if it would have
    > 100% cache hit ratio, because there seem to
    > be circa 4ms overhead for every access,
    > making it slower 2 times compared my average
    > harddisk access.
    >
    > Part of the overhead is buffer copy to shared
    > memory, maybe the current copy implementation
    > to/from shared memory could improved, but
    > major slowdown still seems to be the process
    > switch.
    >
    > Any idea for faster communication?


    How does you make the memory allocation for the shared memory? Reduce
    the number of times you have to DosAlloc.../DosFree...

    Reduce the number of calls to Dos...Sem... to an absolute minimum.
    1. when you knows you have more than one record to send send all, then
    tick the semaphore once.

    2. When you gets awaken reset the semaphore and work until all data is
    eaten.
    In extreme you may even eat more data as sended with one tick - so be
    aware of getting awaken with nothing to do.

    When the receiver of the data is a PM application you can simple post
    PM messages pointing to the shared memory block conaining the data.
    For that the sender needs NOT to be a PM application itself.

    But real speed up would occure if you have a chance to integrate both
    sender and receiver into the same process - using multithreading.
    Process change is a high time intensive thing because full context
    change is needed. thread change is cheap in CPU usage as the context
    change is minimal.

    >
    > library functions that use

    what? Your message is incomplete.

    --
    Tschau/Bye
    Herbert

    Visit http://www.ecomstation.de the home of german eComStation
    eComStation 1.2 Deutsch ist da!

  3. Re: Fast inter process communication


    "Veit Kannegieser" wrote in message
    news:QDILePYdqOVx-pn2-t2Qc9EF3HdUC@ID-48912.user.individual.de...
    > Currently using named shared memory and
    > Dos32(Wait|Post)EvenSem to filter file access.
    >
    > The purpose of the communication is to provide
    > cache and prefetch functions.
    > The current implementation is slow even if it would have
    > 100% cache hit ratio, because there seem to
    > be circa 4ms overhead for every access,
    > making it slower 2 times compared my average
    > harddisk access.


    The timings are probably on a Pentium4? Changes to the P4 made callgates
    much slower than they used to be,
    so you're probably getting slammed on the eventsem transition to ring 0.
    Write your own semaphore that
    only does a Dos32.Sem call when it really has to. It's not too hard to do.



  4. Re: Fast inter process communication

    Veit Kannegieser wrote:
    > Currently using named shared memory and
    > Dos32(Wait|Post)EvenSem to filter file access.
    >
    > The purpose of the communication is to provide
    > cache and prefetch functions.
    > The current implementation is slow even if it would have
    > 100% cache hit ratio, because there seem to
    > be circa 4ms overhead for every access,
    > making it slower 2 times compared my average
    > harddisk access.


    4ms can't possibly be blamed on the semaphore access itself. If that
    were the case, then my frame rate regulation for some of the programs
    I've written would never work, or would have noticable chop and
    desynchronization with the audio. It has to be in the copy operation.

    Why not just use the shared memory as the transfer area directly,
    instead of copying it to shared memory?

    I see Scott's response, but I just can't imagine *that* much overhead.

    --
    [Reverse the parts of the e-mail address to reply.]


  5. Re: Fast inter process communication


    "Marty" wrote in message
    news:u--dnXDKkccmKSbcRVn-gQ@comcast.com...
    >
    > 4ms can't possibly be blamed on the semaphore access itself. If that

    ....
    > I see Scott's response, but I just can't imagine *that* much overhead.

    Well, that's probably true -- I forget what it was, but 4ms does, in fact,
    seem like a lot.
    However, if the current design causes a task switch, that could gum things
    up.

    -Scott



  6. Re: Fast inter process communication

    Scott and Marty wrote:

    > > 4ms can't possibly be blamed on the semaphore access itself. If that


    Yes. My error was subtracting different values.
    In a clean simple client/server i get 0.04 ms per switch+back,
    on an Intel P3/500.

    > However, if the current design causes a task switch, that could gum things
    > up.


    Most of the time is really used in memory copy..

    * Filling 512K application buffer takes 2.5 ms.
    * Filling 512K shared memory takes 2.7 ms.
    * Reading 512K to shared memory and copying it to the
    application buffer takes 11.8 ms.
    * Reading 512K to application buffer and copying it to the
    shared memory takes 8.5 ms.

    Reading means here from a large FAT filesystem cache.
    The copy/fill procedure is only 386-style code,
    maybe there is small room for improvement.

    if the application disk buffer is gettable shared memory,
    i could skip the copy operation.

    --
    Veit Kannegieser

+ Reply to Thread