Multiple CPUs Process scheduling - Slackware

This is a discussion on Multiple CPUs Process scheduling - Slackware ; It is not possible to force a process to use a particular core or CPU. Is there a way to influence the kernel's scheduler to at least improve the chance that a process will be run on a particular core ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: Multiple CPUs Process scheduling

  1. Multiple CPUs Process scheduling

    It is not possible to force a process to use a particular core or CPU.
    Is there a way to influence the kernel's scheduler to at least improve
    the chance that a process will be run on a particular core or CPU?

    The default scheduler in the kernel is too stupid to select the least
    used core when a new process is started. I've checked Oskar
    Andreassen's docs and done some googleing and not found anything
    promising, so I'm hoping someone here can help.

    It is really frustrating to have a second instance of dd (run from a
    different tty) be run on CPU0 when there is already a dd running on
    CPU0 that is using at least 75% of CPU0's resources (per mpstat).
    --
    buck

  2. Re: Multiple CPUs Process scheduling

    buck wrote:
    > It is not possible to force a process to use a particular core or CPU.
    > Is there a way to influence the kernel's scheduler to at least improve
    > the chance that a process will be run on a particular core or CPU?
    >
    > The default scheduler in the kernel is too stupid to select the least
    > used core when a new process is started. I've checked Oskar
    > Andreassen's docs and done some googleing and not found anything
    > promising, so I'm hoping someone here can help.
    >
    > It is really frustrating to have a second instance of dd (run from a
    > different tty) be run on CPU0 when there is already a dd running on
    > CPU0 that is using at least 75% of CPU0's resources (per mpstat).
    > --
    > buck


    Well you don't say what kernel you're using or what processor(s) you
    have, but what you're experiencing doesn't sound right. For example, if
    you have multicore CPU(s) do you have multicore scheduling support
    enabled in your kernel?

  3. Re: Multiple CPUs Process scheduling

    buck wrote:
    > It is really frustrating to have a second instance of dd (run from a
    > different tty) be run on CPU0 when there is already a dd running on
    > CPU0 that is using at least 75% of CPU0's resources (per mpstat).


    As far as I understand a second instance of "the same program" will
    use the same core as they share code so will use the same core's
    internal cache (code, being read-only and often quite sequentially
    used is very efficiently cached, using the "real" memory REALLY
    slows down a program).
    A different program has a much better chance to run on "the other
    core" as the only code IT shares with the first (dd, in your case) are
    shared object routines (like glibc) and those mostly will be needed in
    both cores anyway, for other processes running IN that core.

    Remember that both cores share the bus to the "real" memory (and the
    motherboard cache), while the cpu-internal cache is split into banks
    for each core. So programs using large amounts of "real" memory
    will not benefit much from using two cores, RAM- and bus-speed (FSB)
    is the limiting factor.
    --
    ************************************************** ******************
    ** Eef Hartman, Delft University of Technology, dept. EWI/TW **
    ** e-mail: E.J.M.Hartman@math.tudelft.nl, fax: +31-15-278 7295 **
    ** snail-mail: P.O. Box 5031, 2600 GA Delft, The Netherlands **
    ************************************************** ******************

  4. Re: Multiple CPUs Process scheduling

    buck wrote:

    > It is not possible to force a process to use a particular core or CPU.
    > Is there a way to influence the kernel's scheduler to at least improve
    > the chance that a process will be run on a particular core or CPU?
    >
    > The default scheduler in the kernel is too stupid to select the least
    > used core when a new process is started. I've checked Oskar
    > Andreassen's docs and done some googleing and not found anything
    > promising, so I'm hoping someone here can help.
    >
    > It is really frustrating to have a second instance of dd (run from a
    > different tty) be run on CPU0 when there is already a dd running on
    > CPU0 that is using at least 75% of CPU0's resources (per mpstat).
    > --
    > buck


    I think the links below may help, but I might have misunderstood the
    problem.

    http://rlove.org/schedutils/schedutils-1.5.0.tar.gz
    http://www.cyberciti.biz/tips/settin...r-process.html
    ** Posted from http://www.teranews.com **

  5. Re: Multiple CPUs Process scheduling

    On Fri, 06 Jun 2008 01:36:26 -0500, Chris Sorenson
    wrote:

    >buck wrote:
    >> The default scheduler in the kernel is too stupid to select the least
    >> used core when a new process is started. I've checked Oskar
    >> Andreassen's docs and done some googleing and not found anything
    >> promising, so I'm hoping someone here can help.
    >> --
    >> buck

    >
    >Well you don't say what kernel you're using or what processor(s) you
    >have, but what you're experiencing doesn't sound right. For example, if
    >you have multicore CPU(s) do you have multicore scheduling support
    >enabled in your kernel?


    I'm at work, so the computer in question is not available, but I can
    tell you that the kernel build is "stock" 12.0 (2.6.21.5), so unless
    Pat failed to include multicore scheduling I have to assume that there
    is a scheduler.
    --
    buck

  6. Re: Multiple CPUs Process scheduling

    On Fri, 06 Jun 2008 09:25:48 +0200, Eef Hartman
    wrote:

    >buck wrote:
    >> It is really frustrating to have a second instance of dd (run from a
    >> different tty) be run on CPU0 when there is already a dd running on
    >> CPU0 that is using at least 75% of CPU0's resources (per mpstat).

    >
    >As far as I understand a second instance of "the same program" will
    >use the same core as they share code so will use the same core's
    >internal cache (code, being read-only and often quite sequentially
    >used is very efficiently cached, using the "real" memory REALLY
    >slows down a program).
    >A different program has a much better chance to run on "the other
    >core" as the only code IT shares with the first (dd, in your case) are
    >shared object routines (like glibc) and those mostly will be needed in
    >both cores anyway, for other processes running IN that core.
    >
    >Remember that both cores share the bus to the "real" memory (and the
    >motherboard cache), while the cpu-internal cache is split into banks
    >for each core. So programs using large amounts of "real" memory
    >will not benefit much from using two cores, RAM- and bus-speed (FSB)
    >is the limiting factor.


    That's a pretty lucid answer, and to a certain degree I can go along
    with the logic. But there is plenty of RAM available for a second
    instance of dd whereas there is only .19% (or so. Thats POINT one
    nine percent) free resources on CPU0, so the second instance really
    needs to run in CPU1.

    So, if I understand correctly, I'm screwed. A second dd will cause
    the running dd to reduce speed to half of what it is, and the two
    running dd instances will then have about 50% each of CPU0. And CPU1
    will continue to sit there doing next to nothing. BAH HUMBUG
    --
    buck

  7. Re: Multiple CPUs Process scheduling

    On Fri, 06 Jun 2008 10:10:17 -0700, Miguel De Anda
    wrote:

    >buck wrote:


    >> Is there a way to influence the kernel's scheduler to at least improve
    >> the chance that a process will be run on a particular core or CPU?
    >> --
    >> buck

    >
    >I think the links below may help, but I might have misunderstood the
    >problem.
    >
    >http://rlove.org/schedutils/schedutils-1.5.0.tar.gz
    >http://www.cyberciti.biz/tips/settin...r-process.html
    >** Posted from http://www.teranews.com **


    Miguel, I think you may have given me the answer. I saw Love's web
    page but missed the schedutilts (now util-linux), where taskset is one
    of the programs. I'll try taskset when I get home.

    Thank You!
    --
    buck

  8. Re: Multiple CPUs Process scheduling

    buck wrote:

    > On Fri, 06 Jun 2008 10:10:17 -0700, Miguel De Anda
    > wrote:
    >
    >>buck wrote:

    >
    >>> Is there a way to influence the kernel's scheduler to at least improve
    >>> the chance that a process will be run on a particular core or CPU?
    >>> --
    >>> buck

    >>
    >>I think the links below may help, but I might have misunderstood the
    >>problem.
    >>
    >>http://rlove.org/schedutils/schedutils-1.5.0.tar.gz
    >>http://www.cyberciti.biz/tips/settin...r-process.html
    >>** Posted from http://www.teranews.com **

    >
    > Miguel, I think you may have given me the answer. I saw Love's web
    > page but missed the schedutilts (now util-linux), where taskset is one
    > of the programs. I'll try taskset when I get home.
    >
    > Thank You!
    > --
    > buck


    Cool. I'm not sure if it works when you have 2 of the same application
    running, it seemed to work for a single process when I tried it. I think I
    was running mencoder and wanted to see how much cpu it used, and not deal
    with stuff bouncing around, or maybe it was java.. either way.


    ** Posted from http://www.teranews.com **

  9. Multiple CPUs Process scheduling - SOLVED by Miguel

    On Fri, 06 Jun 2008 22:31:52 -0700, Miguel De Anda
    wrote:

    >buck wrote:
    >
    >> On Fri, 06 Jun 2008 10:10:17 -0700, Miguel De Anda
    >> wrote:
    >>
    >>>buck wrote:

    >>
    >>>> Is there a way to influence the kernel's scheduler to at least improve
    >>>> the chance that a process will be run on a particular core or CPU?
    >>>> --
    >>>> buck
    >>>
    >>>I think the links below may help, but I might have misunderstood the
    >>>problem.
    >>>
    >>>http://rlove.org/schedutils/schedutils-1.5.0.tar.gz
    >>>http://www.cyberciti.biz/tips/settin...r-process.html
    >>>** Posted from http://www.teranews.com **

    >>
    >> Miguel, I think you may have given me the answer. I saw Love's web
    >> page but missed the schedutilts (now util-linux), where taskset is one
    >> of the programs. I'll try taskset when I get home.
    >>
    >> Thank You!
    >> --
    >> buck

    >
    >Cool. I'm not sure if it works when you have 2 of the same application
    >running, it seemed to work for a single process when I tried it. I think I
    >was running mencoder and wanted to see how much cpu it used, and not deal
    >with stuff bouncing around, or maybe it was java.. either way.


    taskset works based (mostly) on PID. Right now I have 2 instances of
    bzip2 running and I got each on its own CPU by running
    taskset -p 0x000001 $PIDa
    taskset -p 0x000002 $PIDb
    where "PIDa" means the pid of the first bzip2 (1819) and PIDb the
    second bzip2 (2045). top says load average is 2.4 2.4 2.3ish and each
    CPU is at 90+% user.

    Note that mpstat does not work on my setup. It says CPU1 is 99+%
    idle. I know better because top's load averages make mpstat a liar.
    Perhaps I'll try a newer version.

    Once again: Thank You Miguel! (can you tell that I'm a Happy Camper?)
    --
    buck

+ Reply to Thread