How do I flush/invalidate the CPU instruction cache? - Questions

This is a discussion on How do I flush/invalidate the CPU instruction cache? - Questions ; I have a dual processor, x86 computer running redhat 9 linux. I am working with a friend, on a compiler/interpreter environment that can dynamically generate machine code to be executed. In order to make the newly generated code work in ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: How do I flush/invalidate the CPU instruction cache?

  1. How do I flush/invalidate the CPU instruction cache?

    I have a dual processor, x86 computer running redhat 9 linux.
    I am working with a friend, on a compiler/interpreter environment
    that can dynamically generate machine code to be executed.
    In order to make the newly generated code work in the system,
    it is necessary to flush the instruction cache.

    On Microsoft windows, you just call

    FlushInstructionCache(GetCurrentProcess(), NULL, 0);

    I have searched, and I can't find a way to do it.
    I was able to find some old docs to do it on mips based linux.

    Any ideas?

    Thanks,
    Jeff


  2. Re: How do I flush/invalidate the CPU instruction cache?

    turbo wrote:
    >
    > I have a dual processor, x86 computer running redhat 9 linux.
    > I am working with a friend, on a compiler/interpreter environment
    > that can dynamically generate machine code to be executed.
    > In order to make the newly generated code work in the system,
    > it is necessary to flush the instruction cache.
    >
    > On Microsoft windows, you just call
    >
    > FlushInstructionCache(GetCurrentProcess(), NULL, 0);
    >
    > I have searched, and I can't find a way to do it.
    > I was able to find some old docs to do it on mips based linux.


    Does these lines from kernel/modules.c help you?

    /* On some machines it is necessary to do something here
    to make the I and D caches consistent. */
    flush_icache_range((unsigned long)mod, (unsigned long)mod + mod->size);

    --
    Kasper Dupont -- der bruger for meget tid paa usenet.
    For sending spam use mailto:aaarep@daimi.au.dk
    Their business was zero and it was shrinking.

  3. Re: How do I flush/invalidate the CPU instruction cache?

    turbo wrote:
    > I have a dual processor, x86 computer running redhat 9 linux.
    > I am working with a friend, on a compiler/interpreter environment
    > that can dynamically generate machine code to be executed.
    > In order to make the newly generated code work in the system,
    > it is necessary to flush the instruction cache.
    >
    > On Microsoft windows, you just call
    >
    > FlushInstructionCache(GetCurrentProcess(), NULL, 0);
    >
    > I have searched, and I can't find a way to do it.
    > I was able to find some old docs to do it on mips based linux.
    >
    > Any ideas?
    >
    > Thanks,
    > Jeff
    >


    There are some routines defined in the linux kernel for cache flushing,
    but under X86 they are defined to compile to nothing. I'm sure you've
    heard this before, but unless the CPU has some sort of problem, user
    space applications really should not need to flush the cache. This is
    not to say it can't be done, but if you want to, it means you will have
    to implement a flush routine in kernel space (this will of course be
    arch specific), export it in some way to user space and call it from
    there. The need to flush the cache also clearly means you can say
    goodbye to easy portability. Perhaps you can expand on why you need to
    flush the cache, and some people can propose some more portable
    workaround/other solutions for your problem?

    HTH
    Neil

    --
    /************************************************** *
    *Neil Horman
    *Software Engineer
    *Red Hat, Inc.
    *nhorman@redhat.com
    ************************************************** */


  4. Re: How do I flush/invalidate the CPU instruction cache?

    Neil Horman wrote:

    > turbo wrote:
    >
    >> I have a dual processor, x86 computer running redhat 9 linux.
    >> I am working with a friend, on a compiler/interpreter environment
    >> that can dynamically generate machine code to be executed.
    >> In order to make the newly generated code work in the system,
    >> it is necessary to flush the instruction cache.
    >>
    >> On Microsoft windows, you just call
    >>
    >> FlushInstructionCache(GetCurrentProcess(), NULL, 0);
    >>
    >> I have searched, and I can't find a way to do it.
    >> I was able to find some old docs to do it on mips based linux.
    >>
    >> Any ideas?
    >>
    >> Thanks,
    >> Jeff
    >>

    >
    > There are some routines defined in the linux kernel for cache flushing,
    > but under X86 they are defined to compile to nothing. I'm sure you've
    > heard this before, but unless the CPU has some sort of problem, user
    > space applications really should not need to flush the cache. This is
    > not to say it can't be done, but if you want to, it means you will have
    > to implement a flush routine in kernel space (this will of course be
    > arch specific), export it in some way to user space and call it from
    > there. The need to flush the cache also clearly means you can say
    > goodbye to easy portability. Perhaps you can expand on why you need to
    > flush the cache, and some people can propose some more portable
    > workaround/other solutions for your problem?
    >
    > HTH
    > Neil


    As I said, this is for a interactive language that is mostly compiled
    and has the ability to dynamically compile to machine code,
    new program source that is integrated into the existing program, and
    continue executing.

    Imaine an interactive basic interpreter, that already is executing a big
    program, and you type in a new procedure, and say to the language
    "compile and run".

    This happens to be for a large, mature language called ICL (a strongly
    typed garbage collected language designed for building large complex
    systems), invented at Caltech as part of a doctoral thesis in the late
    1970's or so, that has around 1 millions lines of code written for it.
    It was developed for the dec-20, ported to the vax, then the dec alpha,
    and currently runs under windows. I am porting it to x86 Gnu/Linux. It
    was used as the backend of the first e-commerce application, mosis.org,
    where I happened to work from 1985-1995.

    Jeff Deifik


  5. Re: How do I flush/invalidate the CPU instruction cache?

    jeff wrote:
    > Neil Horman wrote:
    >
    >> turbo wrote:
    >>
    >>> I have a dual processor, x86 computer running redhat 9 linux.
    >>> I am working with a friend, on a compiler/interpreter environment
    >>> that can dynamically generate machine code to be executed.
    >>> In order to make the newly generated code work in the system,
    >>> it is necessary to flush the instruction cache.
    >>>
    >>> On Microsoft windows, you just call
    >>>
    >>> FlushInstructionCache(GetCurrentProcess(), NULL, 0);
    >>>
    >>> I have searched, and I can't find a way to do it.
    >>> I was able to find some old docs to do it on mips based linux.
    >>>
    >>> Any ideas?
    >>>
    >>> Thanks,
    >>> Jeff
    >>>

    >>
    >> There are some routines defined in the linux kernel for cache
    >> flushing, but under X86 they are defined to compile to nothing. I'm
    >> sure you've heard this before, but unless the CPU has some sort of
    >> problem, user space applications really should not need to flush the
    >> cache. This is not to say it can't be done, but if you want to, it
    >> means you will have to implement a flush routine in kernel space (this
    >> will of course be arch specific), export it in some way to user space
    >> and call it from there. The need to flush the cache also clearly
    >> means you can say goodbye to easy portability. Perhaps you can expand
    >> on why you need to flush the cache, and some people can propose some
    >> more portable workaround/other solutions for your problem?
    >>
    >> HTH
    >> Neil

    >
    >
    > As I said, this is for a interactive language that is mostly compiled
    > and has the ability to dynamically compile to machine code,
    > new program source that is integrated into the existing program, and
    > continue executing.
    >
    > Imaine an interactive basic interpreter, that already is executing a big
    > program, and you type in a new procedure, and say to the language
    > "compile and run".
    >
    > This happens to be for a large, mature language called ICL (a strongly
    > typed garbage collected language designed for building large complex
    > systems), invented at Caltech as part of a doctoral thesis in the late
    > 1970's or so, that has around 1 millions lines of code written for it.
    > It was developed for the dec-20, ported to the vax, then the dec alpha,
    > and currently runs under windows. I am porting it to x86 Gnu/Linux. It
    > was used as the backend of the first e-commerce application, mosis.org,
    > where I happened to work from 1985-1995.
    >
    > Jeff Deifik
    >

    And what specific problem are you running into that you believe will be
    solved by flushing the cache?
    Neil

    --
    /************************************************** *
    *Neil Horman
    *Software Engineer
    *Red Hat, Inc.
    *nhorman@redhat.com
    ************************************************** */


  6. Re: How do I flush/invalidate the CPU instruction cache?

    jeff writes:
    >
    > As I said, this is for a interactive language that is mostly compiled
    > and has the ability to dynamically compile to machine code,
    > new program source that is integrated into the existing program, and
    > continue executing.


    You may wish to look at:

    http://developer.intel.com/design/pe...als/245472.htm

    where you can download the IA-32 Architecture System Programming
    Guide. Sections 7.1.3 and 10.6 address caching issues having to do
    with self-modifying code.

    Section 7.1.3 lays out the processor synchronization algorithm that
    should be implemented to successfully have one processor modify
    another processor's code in an SMP setup on present and future IA-32
    chips.

    If the same processor is modifying its own code, then all you need to
    do is ensure that a jump occurs somewhere between the modification and
    the execution of the modified code.

    In practice, I believe Section 7.1.3 partially overstates and
    partially misstates the case. For a 486, a jump instruction is
    necessary to flush its prefetch cache, as explained in Section 10.6
    (though 7.1.3 incorrectly contradicts this). For a 386, which has no
    on-chip cache, no action is needed, of course. For post-486 chips in
    a single processor machine, invalidation of all caches is done
    automatically, so no action is needed, either.

    Even in SMP rigs with cross-modifying code, write ordering and bus
    snooping should imply that when the second processor sees the flag
    indicating that the code has been modified, it will have a consistent
    and correct "view" of the modified code so no synchronization
    instruction (CPUID or IRET) is actually needed.

    --
    Kevin

  7. Re: How do I flush/invalidate the CPU instruction cache?

    Neil Horman wrote:
    > jeff wrote:
    >
    >> Neil Horman wrote:
    >>
    >>> turbo wrote:
    >>>
    >>>> I have a dual processor, x86 computer running redhat 9 linux.
    >>>> I am working with a friend, on a compiler/interpreter environment
    >>>> that can dynamically generate machine code to be executed.
    >>>> In order to make the newly generated code work in the system,
    >>>> it is necessary to flush the instruction cache.
    >>>>
    >>>> On Microsoft windows, you just call
    >>>>
    >>>> FlushInstructionCache(GetCurrentProcess(), NULL, 0);
    >>>>
    >>>> I have searched, and I can't find a way to do it.
    >>>> I was able to find some old docs to do it on mips based linux.
    >>>>
    >>>> Any ideas?
    >>>>
    >>>> Thanks,
    >>>> Jeff
    >>>>
    >>>
    >>> There are some routines defined in the linux kernel for cache
    >>> flushing, but under X86 they are defined to compile to nothing. I'm
    >>> sure you've heard this before, but unless the CPU has some sort of
    >>> problem, user space applications really should not need to flush the
    >>> cache. This is not to say it can't be done, but if you want to, it
    >>> means you will have to implement a flush routine in kernel space
    >>> (this will of course be arch specific), export it in some way to user
    >>> space and call it from there. The need to flush the cache also
    >>> clearly means you can say goodbye to easy portability. Perhaps you
    >>> can expand on why you need to flush the cache, and some people can
    >>> propose some more portable workaround/other solutions for your problem?
    >>>
    >>> HTH
    >>> Neil

    >>
    >>
    >>
    >> As I said, this is for a interactive language that is mostly compiled
    >> and has the ability to dynamically compile to machine code,
    >> new program source that is integrated into the existing program, and
    >> continue executing.
    >>
    >> Imaine an interactive basic interpreter, that already is executing a big
    >> program, and you type in a new procedure, and say to the language
    >> "compile and run".
    >>
    >> This happens to be for a large, mature language called ICL (a strongly
    >> typed garbage collected language designed for building large complex
    >> systems), invented at Caltech as part of a doctoral thesis in the late
    >> 1970's or so, that has around 1 millions lines of code written for it.
    >> It was developed for the dec-20, ported to the vax, then the dec
    >> alpha, and currently runs under windows. I am porting it to x86
    >> Gnu/Linux. It was used as the backend of the first e-commerce
    >> application, mosis.org, where I happened to work from 1985-1995.
    >>
    >> Jeff Deifik
    >>

    > And what specific problem are you running into that you believe will be
    > solved by flushing the cache?
    > Neil


    Well, I am dynamically generating new code at memory addresses that
    used to have old code. As the program is executing the cache may
    contain the old code at the modified memory addresses. When the cache
    is flushed, the memory is guaranteed to be consistent between the
    cache and main memory.

    This isn't something that is done very often, just after compiling new
    code. If it is not done when run with microsoft windows OS's the
    instructions have gotten corrupted. It is not something that is 100%
    repeatable, likely due to all the other stuff accessing the cache.

    thanks,
    jeff deifik


  8. Re: How do I flush/invalidate the CPU instruction cache?

    jeff wrote:

    > Neil Horman wrote:
    >
    >> jeff wrote:
    >>
    >>> Neil Horman wrote:
    >>>
    >>>> turbo wrote:
    >>>>
    >>>>> I have a dual processor, x86 computer running redhat 9 linux.
    >>>>> I am working with a friend, on a compiler/interpreter environment
    >>>>> that can dynamically generate machine code to be executed.
    >>>>> In order to make the newly generated code work in the system,
    >>>>> it is necessary to flush the instruction cache.
    >>>>>
    >>>>> On Microsoft windows, you just call
    >>>>>
    >>>>> FlushInstructionCache(GetCurrentProcess(), NULL, 0);
    >>>>>
    >>>>> I have searched, and I can't find a way to do it.
    >>>>> I was able to find some old docs to do it on mips based linux.
    >>>>>
    >>>>> Any ideas?
    >>>>>
    >>>>> Thanks,
    >>>>> Jeff
    >>>>>
    >>>>
    >>>> There are some routines defined in the linux kernel for cache
    >>>> flushing, but under X86 they are defined to compile to nothing. I'm
    >>>> sure you've heard this before, but unless the CPU has some sort of
    >>>> problem, user space applications really should not need to flush the
    >>>> cache. This is not to say it can't be done, but if you want to, it
    >>>> means you will have to implement a flush routine in kernel space
    >>>> (this will of course be arch specific), export it in some way to
    >>>> user space and call it from there. The need to flush the cache also
    >>>> clearly means you can say goodbye to easy portability. Perhaps you
    >>>> can expand on why you need to flush the cache, and some people can
    >>>> propose some more portable workaround/other solutions for your problem?
    >>>>
    >>>> HTH
    >>>> Neil
    >>>
    >>>
    >>>
    >>>
    >>> As I said, this is for a interactive language that is mostly compiled
    >>> and has the ability to dynamically compile to machine code,
    >>> new program source that is integrated into the existing program, and
    >>> continue executing.
    >>>
    >>> Imaine an interactive basic interpreter, that already is executing a big
    >>> program, and you type in a new procedure, and say to the language
    >>> "compile and run".
    >>>
    >>> This happens to be for a large, mature language called ICL (a strongly
    >>> typed garbage collected language designed for building large complex
    >>> systems), invented at Caltech as part of a doctoral thesis in the
    >>> late 1970's or so, that has around 1 millions lines of code written
    >>> for it. It was developed for the dec-20, ported to the vax, then the
    >>> dec alpha, and currently runs under windows. I am porting it to x86
    >>> Gnu/Linux. It was used as the backend of the first e-commerce
    >>> application, mosis.org, where I happened to work from 1985-1995.
    >>>
    >>> Jeff Deifik
    >>>

    >> And what specific problem are you running into that you believe will
    >> be solved by flushing the cache?
    >> Neil

    >
    >
    > Well, I am dynamically generating new code at memory addresses that
    > used to have old code. As the program is executing the cache may
    > contain the old code at the modified memory addresses. When the cache
    > is flushed, the memory is guaranteed to be consistent between the
    > cache and main memory.
    >
    > This isn't something that is done very often, just after compiling new
    > code. If it is not done when run with microsoft windows OS's the
    > instructions have gotten corrupted. It is not something that is 100%
    > repeatable, likely due to all the other stuff accessing the cache.
    >
    > thanks,
    > jeff deifik
    >


    I have to correct myself here. The corruption was observed running
    VMS on vaxstations II, and vaxstation III's. The call to flush the cache
    was added to the system at that time @1990, and was kept in place under
    Microsoft Windows. I will check if the system can run without the cache
    flush wunder Microsoft Windows.

    Jeff Deifik


  9. Re: How do I flush/invalidate the CPU instruction cache?

    In comp.os.linux.development.system jeff wrote:
    > Well, I am dynamically generating new code at memory addresses that
    > used to have old code. As the program is executing the cache may
    > contain the old code at the modified memory addresses. When the cache
    > is flushed, the memory is guaranteed to be consistent between the
    > cache and main memory.


    Well, you must be running code in odd places, because the
    usual code pages (.text) are read-only to permit sharing.

    This is an _old_ problem, generaly known on x86 as "Self
    Modifying Code". x86 has had to handle this well because
    old MS-DOS programs sometimes used such tricks.

    In general, it is possible and the CPU will get the right
    (just written) code so long as the code being modified is _NOT_
    on the current cacheline being executed. Performance of
    such code is terrible, and getting worse.

    There is also a Ring0 instruction `wbinvd` that will flush
    the caches, but it is _horribly_ long and more used when the
    CPU will power-off into battery RAM.

    -- Robert



+ Reply to Thread