seeking highly efficient caches scheme for demanding engineering computing? - Programmer

This is a discussion on seeking highly efficient caches scheme for demanding engineering computing? - Programmer ; seeking highly efficient caches scheme for demanding engineering computing? HI all, To same the time of costly function evaluation, I want to explore the possibility of caching. Suppose in millions of calls to the function evaluation, there are some computations, ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: seeking highly efficient caches scheme for demanding engineering computing?

  1. seeking highly efficient caches scheme for demanding engineering computing?


    seeking highly efficient caches scheme for demanding engineering computing?

    HI all,

    To same the time of costly function evaluation, I want to explore the
    possibility of caching.

    Suppose in millions of calls to the function evaluation, there are some
    computations, in which only a portion of the parameters change, others
    remain the same as the parameters that are used in some other function
    evaluations some time before. To save time, I can group the varying
    parameters into sub-groups, and cache the results for later use. This needs
    a highly efficient cache and a efficient organzation and look-up. Also, the
    decision of whether a group of parameters has been seen before should be
    based on data trunk in binary form, instead of decimal formats, so I can
    compare memory data trunks directly using memory comparison.

    Does anybody have good suggestions and pointers on this approach?

    Thanks!








  2. Re: seeking highly efficient caches scheme for demanding engineering computing?

    On Fri, 27 Jul 2007 00:12:57 -0400, "Luna Moon"
    wrote:

    >
    >seeking highly efficient caches scheme for demanding engineering computing?
    >
    >HI all,
    >
    >To same the time of costly function evaluation, I want to explore the
    >possibility of caching.
    >
    >Suppose in millions of calls to the function evaluation, there are some
    >computations, in which only a portion of the parameters change, others
    >remain the same as the parameters that are used in some other function
    >evaluations some time before. To save time, I can group the varying
    >parameters into sub-groups, and cache the results for later use. This needs
    >a highly efficient cache and a efficient organzation and look-up. Also, the
    >decision of whether a group of parameters has been seen before should be
    >based on data trunk in binary form, instead of decimal formats, so I can
    >compare memory data trunks directly using memory comparison.
    >
    >Does anybody have good suggestions and pointers on this approach?
    >
    >Thanks!



    The problem with all caching schemes is determining whether
    you have the required data in the cache, and fetching it or
    updating it.

    There are hardware solutions to help with CPU and disk caching, but
    in your case you would be doing this all yourself. So it would
    seem this is only feasible if the savings in computation time
    is much more than the cost in housekeeping overhead.
    Seems doubtful for most things...

    Best regards,


    Bob Masta

    D A Q A R T A
    Data AcQuisition And Real-Time Analysis
    www.daqarta.com
    Scope, Spectrum, Spectrogram, Signal Generator
    Science with your sound card!


  3. Re: seeking highly efficient caches scheme for demanding engineering computing?

    On Fri, 27 Jul 2007 13:27:00 GMT, NoSpam@daqarta.com (Bob Masta)
    wrote:

    >On Fri, 27 Jul 2007 00:12:57 -0400, "Luna Moon"
    >wrote:
    >
    >>
    >>seeking highly efficient caches scheme for demanding engineering computing?
    >>
    >>HI all,
    >>
    >>To same the time of costly function evaluation, I want to explore the
    >>possibility of caching.
    >>
    >>Suppose in millions of calls to the function evaluation, there are some
    >>computations, in which only a portion of the parameters change, others
    >>remain the same as the parameters that are used in some other function
    >>evaluations some time before. To save time, I can group the varying
    >>parameters into sub-groups, and cache the results for later use. This needs
    >>a highly efficient cache and a efficient organzation and look-up. Also, the
    >>decision of whether a group of parameters has been seen before should be
    >>based on data trunk in binary form, instead of decimal formats, so I can
    >>compare memory data trunks directly using memory comparison.
    >>
    >>Does anybody have good suggestions and pointers on this approach?
    >>
    >>Thanks!

    >
    >
    >The problem with all caching schemes is determining whether
    >you have the required data in the cache, and fetching it or
    >updating it.
    >
    >There are hardware solutions to help with CPU and disk caching, but
    >in your case you would be doing this all yourself. So it would
    >seem this is only feasible if the savings in computation time
    >is much more than the cost in housekeeping overhead.
    >Seems doubtful for most things...
    >


    On further thought, your original description is a pretty
    good description of the Fast Fourier Transform. It
    accomplishes the same ultimate calculation as the
    Discrete Fourier Transform, but it avoids redundant
    computation of sub-products. It doesn't use a specific
    cache structure, but accomplishes the same result.

    So you might want to apply the same sort of strategy
    to your computation. Analyze the heck out of it, and
    figure out where those redundant portions are, and
    rearrange things to use the partial results most efficiently.
    In the FFT case that means doing things in an order
    different from the "obvious" DFT order. You may find
    it helpful to use arrays of indices that point to what
    sub-results to use when, and where to put the results.

    Best regards,




    Bob Masta

    D A Q A R T A
    Data AcQuisition And Real-Time Analysis
    www.daqarta.com
    Scope, Spectrum, Spectrogram, Signal Generator
    Science with your sound card!


  4. Re: seeking highly efficient caches scheme for demanding engineering computing?

    In article ,
    "Luna Moon" writes:
    >
    > seeking highly efficient caches scheme for demanding engineering computing?
    >
    > HI all,
    >
    > To same the time of costly function evaluation, I want to explore the
    > possibility of caching.
    >

    ....
    >
    > Does anybody have good suggestions and pointers on this approach?


    http://en.wikipedia.org/wiki/Dynamic_programming is a good starting
    point.


+ Reply to Thread