[9fans] 9bench: lmbench for Plan 9 - Plan9

This is a discussion on [9fans] 9bench: lmbench for Plan 9 - Plan9 ; I've just finished porting what we thought were the 8 most useful benchmarks from the lmbench v.2 suite to run natively in Plan 9. The benchmarks are: bw_file_rd: Measure file read bandwidth bw_mem: Measure memory bandwidth for a several different ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: [9fans] 9bench: lmbench for Plan 9

  1. [9fans] 9bench: lmbench for Plan 9

    I've just finished porting what we thought were the 8 most useful
    benchmarks from the lmbench v.2 suite to run natively in Plan 9. The
    benchmarks are:
    bw_file_rd: Measure file read bandwidth
    bw_mem: Measure memory bandwidth for a several different operations
    bw_pipe: Measure pipe bandwidth
    bw_tcp: Measure tcp bandwidth on the local machine
    lat_ctx: Measure context switch latency
    lat_syscall: Measure system call latency
    lat_tcp: Measure tcp latency on local system
    lat_udp: Measure udp latency on local system

    As I note in the README, lmbench's license carries the additional
    restriction that you cannot publish results from modified benchmarks,
    so keep your results to yourself.

    The source is in /n/sources/contrib/john/9bench.tgz


    John



  2. Re: [9fans] 9bench: lmbench for Plan 9

    > As I note in the README, lmbench's license carries the additional
    > restriction that you cannot publish results from modified benchmarks,
    > so keep your results to yourself.


    doesn't generating unpublishable results defeat the purpose of a benchmark?

    - erik


  3. Re: [9fans] 9bench: lmbench for Plan 9

    On Fri, Mar 21, 2008 at 10:51 AM, erik quanstrom wrote:
    > > As I note in the README, lmbench's license carries the additional
    > > restriction that you cannot publish results from modified benchmarks,
    > > so keep your results to yourself.

    >
    > doesn't generating unpublishable results defeat the purpose of a benchmark?


    what is the purpose?

    ron


  4. Re: [9fans] 9bench: lmbench for Plan 9

    > On Fri, Mar 21, 2008 at 10:51 AM, erik quanstrom wrote:
    > > > As I note in the README, lmbench's license carries the additional
    > > > restriction that you cannot publish results from modified benchmarks,
    > > > so keep your results to yourself.

    > >
    > > doesn't generating unpublishable results defeat the purpose of a benchmark?

    >
    > what is the purpose?


    i would assume that the main reason to go with an official benchmark is to
    compare results. if you don't need to compare results and you want to tell
    if something you've changed has made things faster or slower, i would think
    rolling one's own would be simple and effective. "time dd ..." is often a great
    relative measure.

    - erik


  5. Re: [9fans] 9bench: lmbench for Plan 9

    Perhaps the copyright owner just wanted to make fun of intellectual property


  6. Re: [9fans] 9bench: lmbench for Plan 9

    Is a port of lmbench considered a modified benchmark?
    I understand you cannot publish results saying it´s the output
    of lmbench if you changed the bechmark, but only because you
    used a ported one...?

    Funny in any case.

    On Sat, Mar 22, 2008 at 12:18 PM, hiro <23hiro@googlemail.com> wrote:
    > Perhaps the copyright owner just wanted to make fun of intellectual property
    >
    >


  7. Re: [9fans] 9bench: lmbench for Plan 9

    > Is a port of lmbench considered a modified benchmark?
    > I understand you cannot publish results saying it´s the output
    > of lmbench if you changed the bechmark, but only because you
    > used a ported one...?
    >
    > Funny in any case.


    you're also not allowed to publish partial results. they have
    really overthought things. they are worried that someone
    will try to look better than they are by publishing partial results.

    - erik



  8. Re: [9fans] 9bench: lmbench for Plan 9

    >> Is a port of lmbench considered a modified benchmark?
    >> I understand you cannot publish results saying it´s the output
    >> of lmbench if you changed the bechmark, but only because you
    >> used a ported one...?
    >>
    >> Funny in any case.

    >
    > you're also not allowed to publish partial results. they have
    > really overthought things. they are worried that someone
    > will try to look better than they are by publishing partial results.
    >
    > - erik


    Apparently he also didn't want companies (read: Sun) rewriting portions
    of the benchmark to make their software/hardware look better.
    Since Plan 9 doesn't have sockets or the bzero call or a number of other
    things that you'll find on Unix, I had to do some rewriting. I made
    sure to accomplish the same tasks as on Unix, but in some cases more
    than others I had to do a lot of changing. The timing code should
    still be pretty much the same, although I added a little to include
    counting of clock cycles (approximate). The pipe bandwidth test, if
    I remember correctly, took basically no changes to port (I think I
    had to change sprintf to sprint or something like that), but others,
    like the networking ones, required that I essentially cut out the
    majority of the tests and replace them with a Plan 9 equivalent. If
    anyone is running these and finds a test that has the wrong behavior,
    please let me know so I can fix it.

    As for the utility of the benchmarks, come on. I know there are people
    on this list who write kernel code (hmm, I seem to recall somebody
    on here who does a lot of work on networking drivers...) and they
    may find it useful to get these kinds of measurements to see if
    changes have improved performance. Besides, there's nothing to stop you
    from running the benchmarks on Linux and Plan 9 and comparing (for
    yourself) the difference in performance--you just can't try for
    e-cred by posting your results on comp.os.linux (hint: the
    comparisons don't come out very favorably for Plan 9 anyway).

    Oh, and if anybody ever wants to see the Macros From Hell, just take
    a look at bench.h for BENCH0, BENCH, BENCH1, and BENCH-INNER.
    (disclaimer: I didn't write those)


    John



  9. Re: [9fans] 9bench: lmbench for Plan 9

    > As for the utility of the benchmarks, come on. I know there are people
    > on this list who write kernel code (hmm, I seem to recall somebody
    > on here who does a lot of work on networking drivers...) and they
    > may find it useful to get these kinds of measurements to see if
    > changes have improved performance. Besides, there's nothing to stop you
    > from running the benchmarks on Linux and Plan 9 and comparing (for
    > yourself) the difference in performance--you just can't try for
    > e-cred by posting your results on comp.os.linux (hint: the
    > comparisons don't come out very favorably for Plan 9 anyway).


    i tend to use very simple timing runs in my testing. i use dd for AoE
    and disk drivers and a small program named zerohose for (other)
    network protcols. time(1) seems good enough for timing given an
    appropriate amount of i/o.

    the reason for the simple timing tests is because good hardware is
    hard to get wrong but marginal hardware (bad design, undersized
    buffers, unsufficient pcie lanes) is typically very sensitve to cache
    pecularities. even with simple tests some hardware can have >20%
    variance between runs. it's very hard to reason about this sort of
    thing. i'd hate to have to also have to understand what the benchmark
    is doing for^wto me, too.

    oh, plan 9 can push packets faster on the nics i've tested than linux,
    but for some reason, plan 9 tcp performance hasn't been very good,
    especially on 10gbe hardware. il and aoe fair much better.

    - erik



+ Reply to Thread