Pthreads + SMP - Unix

This is a discussion on Pthreads + SMP - Unix ; I have a couple of general questions about threads on Linux (2.6 kernels) on SMP systems (with kernel SMP support) -- using POSIX threads: 1) Just to confirm, the Linux kernel will assign a process to any CPU in the ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: Pthreads + SMP

  1. Pthreads + SMP

    I have a couple of general questions about threads on Linux (2.6
    kernels) on SMP systems (with kernel SMP support) -- using POSIX
    threads:

    1) Just to confirm, the Linux kernel will assign a process to any CPU
    in the system, and I can control the per-process CPU mask with
    sched_setaffinity, right?

    2) Can threads created with pthread_create run on any CPU (that is in
    the process's CPU affinity mask, of course)?

    3) Is there a way to set the CPU affinity mask per-thread?

    4) Will the kernel ever move a process to a different CPU than the one
    that it started on, or is it safe to say that if a process starts on a
    given CPU, it will remain on that CPU until it terminates?

    Thanks,
    Jason

  2. Re: Pthreads + SMP

    "jason.cipriani@gmail.com" writes:
    >I have a couple of general questions about threads on Linux (2.6
    >kernels) on SMP systems (with kernel SMP support) -- using POSIX
    >threads:
    >
    >1) Just to confirm, the Linux kernel will assign a process to any CPU
    >in the system, and I can control the per-process CPU mask with
    >sched_setaffinity, right?


    yes.

    >
    >2) Can threads created with pthread_create run on any CPU (that is in
    >the process's CPU affinity mask, of course)?


    yes.

    >
    >3) Is there a way to set the CPU affinity mask per-thread?


    From the linux man page:

    The affinity mask is actually a per-thread attribute that can be
    adjusted independently for each of the threads in a thread group. The
    value returned from a call to gettid(2) can be passed in the argument
    pid.
    >
    >4) Will the kernel ever move a process to a different CPU than the one
    >that it started on, or is it safe to say that if a process starts on a
    >given CPU, it will remain on that CPU until it terminates?


    Absent an affinity mask, the processor may move a thread to a different
    CPU (a process is a single thread). The processor will attempt to keep
    the thread on a CPU once it has been scheduled on that CPU to avoid
    cold-cache effects, but there are no guarantees.

    scott

  3. Re: Pthreads + SMP

    Thanks, that answers all of my questions! I missed the part about
    passing gettid() to sched_setaffinity() in the man page, thanks for
    digging that up.

    Jason

    On Mar 5, 6:14 pm, sc...@slp53.sl.home (Scott Lurndal) wrote:
    > "jason.cipri...@gmail.com" writes:
    > >I have a couple of general questions about threads on Linux (2.6
    > >kernels) on SMP systems (with kernel SMP support) -- using POSIX
    > >threads:

    >
    > >1) Just to confirm, the Linux kernel will assign a process to any CPU
    > >in the system, and I can control the per-process CPU mask with
    > >sched_setaffinity, right?

    >
    > yes.
    >
    >
    >
    > >2) Can threads created with pthread_create run on any CPU (that is in
    > >the process's CPU affinity mask, of course)?

    >
    > yes.
    >
    >
    >
    > >3) Is there a way to set the CPU affinity mask per-thread?

    >
    > From the linux man page:
    >
    > The affinity mask is actually a per-thread attribute that can be
    > adjusted independently for each of the threads in a thread group. The
    > value returned from a call to gettid(2) can be passed in the argument
    > pid.
    >
    >
    >
    > >4) Will the kernel ever move a process to a different CPU than the one
    > >that it started on, or is it safe to say that if a process starts on a
    > >given CPU, it will remain on that CPU until it terminates?

    >
    > Absent an affinity mask, the processor may move a thread to a different
    > CPU (a process is a single thread). The processor will attempt to keep
    > the thread on a CPU once it has been scheduled on that CPU to avoid
    > cold-cache effects, but there are no guarantees.
    >
    > scott



  4. Re: Pthreads + SMP

    Oops, one more quick question:

    On Mar 5, 6:14 pm, sc...@slp53.sl.home (Scott Lurndal) wrote:
    > >3) Is there a way to set the CPU affinity mask per-thread?

    >
    > From the linux man page:
    >
    > The affinity mask is actually a per-thread attribute that can be
    > adjusted independently for each of the threads in a thread group. The
    > value returned from a call to gettid(2) can be passed in the argument
    > pid.


    Since it is per-thread; that means that the affinity mask for each
    thread does *not* have to be a subset of the owning process's affinity
    mask, right (on Windows, thread affinity masks must be a subset of the
    process's affinity mask)? The "process" affinity mask on Linux is more
    like, the affinity mask of the process's main thread?

    Thanks,
    Jason

  5. Re: Pthreads + SMP

    On Mar 5, 2:40 pm, "jason.cipri...@gmail.com"
    wrote:

    > 4) Will the kernel ever move a process to a different CPU than the one
    > that it started on, or is it safe to say that if a process starts on a
    > given CPU, it will remain on that CPU until it terminates?


    The kernel will constantly move processes across CPUs. Suppose four
    processes start running on a machine with two processors and they all
    burn CPU. Call them A, B, C, and D. The kernel has no idea how long
    those processes will continue to run. Now suppose two of them later
    terminate. If the two that are still running were on the same CPU,
    what can the kernel do other than migrate one of them?

    DS

  6. Re: Pthreads + SMP

    On Mar 5, 4:18 pm, "jason.cipri...@gmail.com"
    wrote:

    > Since it is per-thread; that means that the affinity mask for each
    > thread does *not* have to be a subset of the owning process's affinity
    > mask, right (on Windows, thread affinity masks must be a subset of the
    > process's affinity mask)? The "process" affinity mask on Linux is more
    > like, the affinity mask of the process's main thread?


    Since the Linux kernel does not schedule processes, processes do not
    need or have an affinity mask on Linux. When a thread is created, it
    typically inherits the affinity mask of the process that created it.
    When a process is created, the thread that created it retains the same
    affinity mask it had.

    DS



  7. Re: Pthreads + SMP

    On Mar 5, 11:13 pm, David Schwartz wrote:
    > On Mar 5, 4:18 pm, "jason.cipri...@gmail.com"
    >
    > wrote:
    > > Since it is per-thread; that means that the affinity mask for each
    > > thread does *not* have to be a subset of the owning process's affinity
    > > mask, right (on Windows, thread affinity masks must be a subset of the
    > > process's affinity mask)? The "process" affinity mask on Linux is more
    > > like, the affinity mask of the process's main thread?

    >
    > Since the Linux kernel does not schedule processes, processes do not
    > need or have an affinity mask on Linux. When a thread is created, it
    > typically inherits the affinity mask of the process that created it.
    > When a process is created, the thread that created it retains the same
    > affinity mask it had.


    Great; thanks guys!

    Jason

+ Reply to Thread