Socket Design Help - Unix

This is a discussion on Socket Design Help - Unix ; Hi, Imagine that i have a vector container, which contains a list of items that need downloading, and i want X amount of these items to download simultaneously. These items are being transfered over a custom protocol, not something standard ...

+ Reply to Thread
Results 1 to 2 of 2

Thread: Socket Design Help

  1. Socket Design Help

    Hi,

    Imagine that i have a vector container, which contains a list of items
    that need downloading, and i want X amount of these items to download
    simultaneously.

    These items are being transfered over a custom protocol, not something
    standard like http so i cant use a pre made library, rather im dealing
    with raw socket operations.

    How do i create a decent stable architecture for this purpose?

    So my vector is going be the thread safe queue object so to speak,
    that my download objects will access, then process the item from the
    vector.

    Problem is whats the best design approach to do this?

    I have been looking at the Boost library for threading, but some
    people have said i can just use asynchronous I/O and eliminate the
    need for threads, but i have no idea how i would implement asio?

    My idea for threading would be to have 1 socket / download object per
    thread, and each thread would lock/unlock the vector queue when
    required. Would this work? or is it over the top?

    I'm really just after a good understanding of the design so i can
    implement it as hassle free as possible because threading is scary .

    This is for linux, so does anyone know if theres any existing code for
    this type of design i could use, or look at for ideas, any library's
    that exist?

    Thanks for any help,

    Jason

  2. Re: Socket Design Help

    Jason writes:
    > Imagine that i have a vector container, which contains a list of items
    > that need downloading, and i want X amount of these items to download
    > simultaneously.


    [...]

    > My idea for threading would be to have 1 socket / download object per
    > thread, and each thread would lock/unlock the vector queue when
    > required. Would this work?


    It should. I would implement this based on a ring buffer with two
    counting semaphores (one for the number of free slots and one for the
    number of queued download requests) and a mutex lock on the receiving
    end (assuming their is one 'producer thread' queueing download
    requests and n 'download threads' doing the actual work). Initially,
    the 'free slots' semaphore would be set to size of the buffer and the
    'queued requests' semaphore to zero. The download threads would then
    all block on the 'queued requests' semaphore. The 'producer thread'
    would then first do a sem_wait on the 'free slots' semaphore and after
    that returns, write a new download request in the 'current' slot (no
    furhter interlocking necessary and do a sem_post on the 'queued
    requests' semaphore. The download threads would be doing a sem_wait on
    that. Everytime this call returns in a download thread, the thread
    would acquire the mutex, remove the current 'to-be-consumed' item from
    the buffer, unlock the mutex, post the 'free slots' semaphore and
    proceed to downloading. After download and download-related processing
    have completed, it would again sem_wait on the queued requests
    semaphore.


+ Reply to Thread