job scheduling - Debian

This is a discussion on job scheduling - Debian ; Hello all, Can someone recommend a good "job scheduling" software, at best with gui and monitoring posibilities. tia stefan -- To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org...

+ Reply to Thread
Results 1 to 13 of 13

Thread: job scheduling

  1. job scheduling

    Hello all,

    Can someone recommend a good "job scheduling" software, at best with gui and
    monitoring posibilities.

    tia
    stefan


    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  2. Re: job scheduling

    Hello

    Am Samstag 06 September 2008 16:50:52 schrieb andrew holway:

    >
    > What are you scheduling?

    bash scripts (jobs)

    tia
    stefan



    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  3. Re: job scheduling

    Hi stefan,

    What are you scheduling?

    ta

    Andy

    On Sat, Sep 6, 2008 at 3:21 PM, Stefan wrote:
    > Hello all,
    >
    > Can someone recommend a good "job scheduling" software, at best with gui and
    > monitoring posibilities.
    >
    > tia
    > stefan
    >
    >
    > --
    > To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    > with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
    >
    >



    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  4. Re: job scheduling

    On 06/09/08 15:54, Stefan wrote:
    > Am Samstag 06 September 2008 16:50:52 schrieb andrew holway:
    >
    >> What are you scheduling?
    >>

    > bash scripts (jobs)
    >


    While Cron has logging facilities (particularly with appropriate outputs
    from your scripts) and GUI options (try here
    ), I'm guessing you are wanting
    something more, could you elaborate?

    Thanks

    Gavin

    --

    Gavin Westwood
    Solutium

    http://www.solutium.net - Going the extra mile to provide a fast,
    helpful, reliable Web Hosting service.



  5. Re: job scheduling

    Hello,
    Am Samstag 06 September 2008 17:07:19 schrieb Gavin Westwood:
    > On 06/09/08 15:54, Stefan wrote:
    > > Am Samstag 06 September 2008 16:50:52 schrieb andrew holway:
    > >> What are you scheduling?

    > >
    > > bash scripts (jobs)

    >
    > While Cron has logging facilities (particularly with appropriate outputs
    > from your scripts) and GUI options (try here
    > ), I'm guessing you are wanting
    > something more, could you elaborate?


    yes , something like this:
    http://www.tidalsoft.com/products/en...cheduling.aspx
    on per open source (a wish, not a must) basis and with mysql database

    tia
    stefan


    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  6. Re: job scheduling

    Still pretty vague as far as features, but take a look at Condor
    and the various links on wikipedias Job
    Scheduler page

    --On September 6, 2008 5:32:49 PM +0200 Stefan
    wrote:

    > Hello,
    > Am Samstag 06 September 2008 17:07:19 schrieb Gavin Westwood:
    >> On 06/09/08 15:54, Stefan wrote:
    >> > Am Samstag 06 September 2008 16:50:52 schrieb andrew holway:
    >> >> What are you scheduling?
    >> >
    >> > bash scripts (jobs)

    >>
    >> While Cron has logging facilities (particularly with appropriate outputs
    >> from your scripts) and GUI options (try here
    >> ), I'm guessing you are wanting
    >> something more, could you elaborate?

    >
    > yes , something like this:
    > http://www.tidalsoft.com/products/en...cheduling.aspx
    > on per open source (a wish, not a must) basis and with mysql database
    >
    > tia
    > stefan
    >
    >
    > --
    > To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    > with a subject of "unsubscribe". Trouble? Contact
    > listmaster@lists.debian.org
    >
    >
    >




    --
    Michael Loftis
    Modwest Operations Manager
    Powerful, Affordable Web Hosting


    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  7. Re: job scheduling

    If youlet us know what your'e trying to do maybe we can help better. There
    are quite a few 'types' of job scheduling structures. some, like condor,
    are designed to try to distribute jobs, some are cluster management systems
    designed to make sure all machines of class X run script Y. etc.

    --On September 6, 2008 5:32:49 PM +0200 Stefan
    wrote:

    > Hello,
    > Am Samstag 06 September 2008 17:07:19 schrieb Gavin Westwood:
    >> On 06/09/08 15:54, Stefan wrote:
    >> > Am Samstag 06 September 2008 16:50:52 schrieb andrew holway:
    >> >> What are you scheduling?
    >> >
    >> > bash scripts (jobs)

    >>
    >> While Cron has logging facilities (particularly with appropriate outputs
    >> from your scripts) and GUI options (try here
    >> ), I'm guessing you are wanting
    >> something more, could you elaborate?

    >
    > yes , something like this:
    > http://www.tidalsoft.com/products/en...cheduling.aspx
    > on per open source (a wish, not a must) basis and with mysql database
    >
    > tia
    > stefan
    >
    >
    > --
    > To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    > with a subject of "unsubscribe". Trouble? Contact
    > listmaster@lists.debian.org
    >
    >
    >




    --
    Michael Loftis
    Modwest Operations Manager
    Powerful, Affordable Web Hosting


    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  8. Re: job scheduling

    Am Samstag 06 September 2008 22:39:13 schrieb Michael Loftis:
    > If youlet us know what your'e trying to do maybe we can help better. There
    > are quite a few 'types' of job scheduling structures. some, like condor,
    > are designed to try to distribute jobs, some are cluster management systems
    > designed to make sure all machines of class X run script Y. etc.


    Yes, Im new to this szenarios. Yes what I need at first are job scheduling
    structures and cluster manangement are also interessting. After abit
    searching I found
    http://jobscheduler.sourceforge.net for job scheduling. That looks very good,
    open source and it supports mysql.

    Im thinking also about a cluster with multiple machines where I can run
    distributed jobs, so Im scaleable for greater processing mechanism.
    btw. Is it better to run batch- jobs on a parallel- Cluster like beowolf?

    tia
    stefan



    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  9. Re: job scheduling



    --On September 7, 2008 12:02:49 PM +0200 Stefan
    wrote:

    > Am Samstag 06 September 2008 22:39:13 schrieb Michael Loftis:
    >> If youlet us know what your'e trying to do maybe we can help better.
    >> There are quite a few 'types' of job scheduling structures. some, like
    >> condor, are designed to try to distribute jobs, some are cluster
    >> management systems designed to make sure all machines of class X run
    >> script Y. etc.

    >
    > Yes, Im new to this szenarios. Yes what I need at first are job
    > scheduling structures and cluster manangement are also interessting.
    > After abit searching I found
    > http://jobscheduler.sourceforge.net for job scheduling. That looks very
    > good, open source and it supports mysql.
    >
    > Im thinking also about a cluster with multiple machines where I can run
    > distributed jobs, so Im scaleable for greater processing mechanism.
    > btw. Is it better to run batch- jobs on a parallel- Cluster like beowolf?


    Very application specific. There's no one answer. For some applications
    it's better to split the workload up into units, distributing units to each
    compute node (rendering computer graphics and animation, certain types of
    number crunching, lend themselves well to this) -- that's for workloads
    that a single job can take a single node days, weeks, or months. This
    requires an application that understands how to be split up like that. In
    workloads where the compute effort on each job is low, but there are many
    jobs, then scheduling a job as a whole makes more sense, spreading otu the
    many jobs across the available compute nodes. Job/Batch scheduling is like
    a non-interactive (or...less interactive) version of load balancing, though
    load balancing is part of job scheduling. Load balancing by itself can
    refer to other things (like HTTP or SMTP transactions being spread out over
    a cluster of identical machines).

    >
    > tia
    > stefan
    >
    >
    >
    > --
    > To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    > with a subject of "unsubscribe". Trouble? Contact
    > listmaster@lists.debian.org
    >
    >
    >




    --
    Michael Loftis
    Modwest Operations Manager
    Powerful, Affordable Web Hosting


    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  10. Re: job scheduling

    Am Sonntag 07 September 2008 20:33:03 schrieb Michael Loftis:
    > --On September 7, 2008 12:02:49 PM +0200 Stefan
    >
    > wrote:
    > > Am Samstag 06 September 2008 22:39:13 schrieb Michael Loftis:
    > >> If youlet us know what your'e trying to do maybe we can help better.
    > >> There are quite a few 'types' of job scheduling structures. some, like
    > >> condor, are designed to try to distribute jobs, some are cluster
    > >> management systems designed to make sure all machines of class X run
    > >> script Y. etc.

    > >
    > > Yes, Im new to this szenarios. Yes what I need at first are job
    > > scheduling structures and cluster manangement are also interessting.
    > > After abit searching I found
    > > http://jobscheduler.sourceforge.net for job scheduling. That looks very
    > > good, open source and it supports mysql.
    > >
    > > Im thinking also about a cluster with multiple machines where I can run
    > > distributed jobs, so Im scaleable for greater processing mechanism.
    > > btw. Is it better to run batch- jobs on a parallel- Cluster like
    > > beowolf?

    >
    > Very application specific. There's no one answer. For some applications
    > it's better to split the workload up into units, distributing units to each
    > compute node (rendering computer graphics and animation, certain types of
    > number crunching, lend themselves well to this) -- that's for workloads
    > that a single job can take a single node days, weeks, or months. This
    > requires an application that understands how to be split up like that. In
    > workloads where the compute effort on each job is low, but there are many
    > jobs, then scheduling a job as a whole makes more sense, spreading otu the
    > many jobs across the available compute nodes. Job/Batch scheduling is like
    > a non-interactive (or...less interactive) version of load balancing, though
    > load balancing is part of job scheduling. Load balancing by itself can
    > refer to other things (like HTTP or SMTP transactions being spread out over
    > a cluster of identical machines).


    okay, thanks. You mean sometimes its better to use load- balancing rather than
    parallel- clustering? Or use it in combination? How to made this disition?
    Are there anywhere some parameters I can use?

    I find condor also a good solution.
    Some of you use "condor" or "jobscheduler" in production and can tell me about
    his/ her experience ?

    tia
    stefan



    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  11. Re: job scheduling



    --On September 12, 2008 7:22:23 PM +0200 Stefan
    wrote:


    > okay, thanks. You mean sometimes its better to use load- balancing rather
    > than parallel- clustering? Or use it in combination? How to made this
    > disition? Are there anywhere some parameters I can use?
    >
    > I find condor also a good solution.
    > Some of you use "condor" or "jobscheduler" in production and can tell me
    > about his/ her experience ?


    Probably not, since that form of job scheduling/management is almost
    universally not applicable to any type of workload an ISP does. You need
    to talk to people in bioinformatics or things like that. ISPs are usually
    mostly concerned with interactive/live load balancing, not job scheduling
    and queuing. If we're queuing jobs we're dead Load balancers are real
    time devices, job schedulers and managers are non-real time devices. The
    former is useful when you have enough compute resources and the jobs are
    all small, the latter when the jobs are all large, or possibly when you
    really do not have enough compute resources to complete the tasks in real
    time, or when there's little or no requirement for interactivity. Very
    different animals from what ISPs use/need do. That said, billing systems
    can be handled by job scheduling/management, so can log analysis. But I
    doubt anyone does that here.

    >
    > tia
    > stefan
    >
    >
    >
    > --
    > To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    > with a subject of "unsubscribe". Trouble? Contact
    > listmaster@lists.debian.org
    >
    >
    >




    --
    Michael Loftis
    Modwest Operations Manager
    Powerful, Affordable Web Hosting


    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  12. Re: job scheduling

    Am Freitag 12 September 2008 20:01:51 schrieb Michael Loftis:

    > time, or when there's little or no requirement for interactivity. Very
    > different animals from what ISPs use/need do. That said, billing systems
    > can be handled by job scheduling/management, so can log analysis. But I
    > doubt anyone does that here.


    Im thinking this is also good for a "single point of managing" scripts in your
    datacenter. Doing backups, database- dumps, movings from one point. Assuimg
    you have 1000 servers, how do you do your controlling/ monitoring your
    scripts?

    tia
    stefan




    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  13. Re: job scheduling

    Am Mittwoch 17 September 2008 19:06:18 schrieb Stefan:
    > Am Freitag 12 September 2008 20:01:51 schrieb Michael Loftis:
    > > time, or when there's little or no requirement for interactivity. Very
    > > different animals from what ISPs use/need do. That said, billing systems
    > > can be handled by job scheduling/management, so can log analysis. But I
    > > doubt anyone does that here.

    >
    > Im thinking this is also good for a "single point of managing" scripts in
    > your datacenter. Doing backups, database- dumps, movings from one point.
    > Assuimg you have 1000 servers, how do you do your controlling/ monitoring
    > your scripts?


    no answer is also an answer

    stefan



    --
    To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

+ Reply to Thread