turbo wrote:
>
> I have a dual processor, x86 computer running redhat 9 linux.
> I am working with a friend, on a compiler/interpreter environment
> that can dynamically generate machine code to be executed.
> In order to make the newly generated code work in the system,
> it is necessary to flush the instruction cache.
>
> On Microsoft windows, you just call
>
> FlushInstructionCache(GetCurrentProcess(), NULL, 0);
>
> I have searched, and I can't find a way to do it.
> I was able to find some old docs to do it on mips based linux.

Does these lines from kernel/modules.c help you?

/* On some machines it is necessary to do something here
to make the I and D caches consistent. */
flush_icache_range((unsigned long)mod, (unsigned long)mod + mod->size);

--
Kasper Dupont -- der bruger for meget tid paa usenet.
For sending spam use mailto:aaarep@daimi.au.dk
Their business was zero and it was shrinking.

turbo wrote:
> I have a dual processor, x86 computer running redhat 9 linux.
> I am working with a friend, on a compiler/interpreter environment
> that can dynamically generate machine code to be executed.
> In order to make the newly generated code work in the system,
> it is necessary to flush the instruction cache.
>
> On Microsoft windows, you just call
>
> FlushInstructionCache(GetCurrentProcess(), NULL, 0);
>
> I have searched, and I can't find a way to do it.
> I was able to find some old docs to do it on mips based linux.
>
> Any ideas?
>
> Thanks,
> Jeff
>

There are some routines defined in the linux kernel for cache flushing,
but under X86 they are defined to compile to nothing. I'm sure you've
heard this before, but unless the CPU has some sort of problem, user
space applications really should not need to flush the cache. This is
not to say it can't be done, but if you want to, it means you will have
to implement a flush routine in kernel space (this will of course be
arch specific), export it in some way to user space and call it from
there. The need to flush the cache also clearly means you can say
goodbye to easy portability. Perhaps you can expand on why you need to
flush the cache, and some people can propose some more portable
workaround/other solutions for your problem?

HTH
Neil

--
/************************************************** *
*Neil Horman
*Software Engineer
*Red Hat, Inc.
*nhorman@redhat.com
************************************************** */

Neil Horman wrote:

> turbo wrote:
>
>> I have a dual processor, x86 computer running redhat 9 linux.
>> I am working with a friend, on a compiler/interpreter environment
>> that can dynamically generate machine code to be executed.
>> In order to make the newly generated code work in the system,
>> it is necessary to flush the instruction cache.
>>
>> On Microsoft windows, you just call
>>
>> FlushInstructionCache(GetCurrentProcess(), NULL, 0);
>>
>> I have searched, and I can't find a way to do it.
>> I was able to find some old docs to do it on mips based linux.
>>
>> Any ideas?
>>
>> Thanks,
>> Jeff
>>
>
> There are some routines defined in the linux kernel for cache flushing,
> but under X86 they are defined to compile to nothing. I'm sure you've
> heard this before, but unless the CPU has some sort of problem, user
> space applications really should not need to flush the cache. This is
> not to say it can't be done, but if you want to, it means you will have
> to implement a flush routine in kernel space (this will of course be
> arch specific), export it in some way to user space and call it from
> there. The need to flush the cache also clearly means you can say
> goodbye to easy portability. Perhaps you can expand on why you need to
> flush the cache, and some people can propose some more portable
> workaround/other solutions for your problem?
>
> HTH
> Neil

As I said, this is for a interactive language that is mostly compiled
and has the ability to dynamically compile to machine code,
new program source that is integrated into the existing program, and
continue executing.

Imaine an interactive basic interpreter, that already is executing a big
program, and you type in a new procedure, and say to the language
"compile and run".

This happens to be for a large, mature language called ICL (a strongly
typed garbage collected language designed for building large complex
systems), invented at Caltech as part of a doctoral thesis in the late
1970's or so, that has around 1 millions lines of code written for it.
It was developed for the dec-20, ported to the vax, then the dec alpha,
and currently runs under windows. I am porting it to x86 Gnu/Linux. It
was used as the backend of the first e-commerce application, mosis.org,
where I happened to work from 1985-1995.

Jeff Deifik

jeff wrote:
> Neil Horman wrote:
>
>> turbo wrote:
>>
>>> I have a dual processor, x86 computer running redhat 9 linux.
>>> I am working with a friend, on a compiler/interpreter environment
>>> that can dynamically generate machine code to be executed.
>>> In order to make the newly generated code work in the system,
>>> it is necessary to flush the instruction cache.
>>>
>>> On Microsoft windows, you just call
>>>
>>> FlushInstructionCache(GetCurrentProcess(), NULL, 0);
>>>
>>> I have searched, and I can't find a way to do it.
>>> I was able to find some old docs to do it on mips based linux.
>>>
>>> Any ideas?
>>>
>>> Thanks,
>>> Jeff
>>>
>>
>> There are some routines defined in the linux kernel for cache
>> flushing, but under X86 they are defined to compile to nothing. I'm
>> sure you've heard this before, but unless the CPU has some sort of
>> problem, user space applications really should not need to flush the
>> cache. This is not to say it can't be done, but if you want to, it
>> means you will have to implement a flush routine in kernel space (this
>> will of course be arch specific), export it in some way to user space
>> and call it from there. The need to flush the cache also clearly
>> means you can say goodbye to easy portability. Perhaps you can expand
>> on why you need to flush the cache, and some people can propose some
>> more portable workaround/other solutions for your problem?
>>
>> HTH
>> Neil
>
>
> As I said, this is for a interactive language that is mostly compiled
> and has the ability to dynamically compile to machine code,
> new program source that is integrated into the existing program, and
> continue executing.
>
> Imaine an interactive basic interpreter, that already is executing a big
> program, and you type in a new procedure, and say to the language
> "compile and run".
>
> This happens to be for a large, mature language called ICL (a strongly
> typed garbage collected language designed for building large complex
> systems), invented at Caltech as part of a doctoral thesis in the late
> 1970's or so, that has around 1 millions lines of code written for it.
> It was developed for the dec-20, ported to the vax, then the dec alpha,
> and currently runs under windows. I am porting it to x86 Gnu/Linux. It
> was used as the backend of the first e-commerce application, mosis.org,
> where I happened to work from 1985-1995.
>
> Jeff Deifik
>
And what specific problem are you running into that you believe will be
solved by flushing the cache?
Neil

--
/************************************************** *
*Neil Horman
*Software Engineer
*Red Hat, Inc.
*nhorman@redhat.com
************************************************** */

jeff writes:
>
> As I said, this is for a interactive language that is mostly compiled
> and has the ability to dynamically compile to machine code,
> new program source that is integrated into the existing program, and
> continue executing.

You may wish to look at:

http://developer.intel.com/design/pe...als/245472.htm

where you can download the IA-32 Architecture System Programming
Guide. Sections 7.1.3 and 10.6 address caching issues having to do
with self-modifying code.

Section 7.1.3 lays out the processor synchronization algorithm that
should be implemented to successfully have one processor modify
another processor's code in an SMP setup on present and future IA-32
chips.

If the same processor is modifying its own code, then all you need to
do is ensure that a jump occurs somewhere between the modification and
the execution of the modified code.

In practice, I believe Section 7.1.3 partially overstates and
partially misstates the case. For a 486, a jump instruction is
necessary to flush its prefetch cache, as explained in Section 10.6
(though 7.1.3 incorrectly contradicts this). For a 386, which has no
on-chip cache, no action is needed, of course. For post-486 chips in
a single processor machine, invalidation of all caches is done
automatically, so no action is needed, either.

Even in SMP rigs with cross-modifying code, write ordering and bus
snooping should imply that when the second processor sees the flag
indicating that the code has been modified, it will have a consistent
and correct "view" of the modified code so no synchronization
instruction (CPUID or IRET) is actually needed.

--
Kevin

Neil Horman wrote:
> jeff wrote:
>
>> Neil Horman wrote:
>>
>>> turbo wrote:
>>>
>>>> I have a dual processor, x86 computer running redhat 9 linux.
>>>> I am working with a friend, on a compiler/interpreter environment
>>>> that can dynamically generate machine code to be executed.
>>>> In order to make the newly generated code work in the system,
>>>> it is necessary to flush the instruction cache.
>>>>
>>>> On Microsoft windows, you just call
>>>>
>>>> FlushInstructionCache(GetCurrentProcess(), NULL, 0);
>>>>
>>>> I have searched, and I can't find a way to do it.
>>>> I was able to find some old docs to do it on mips based linux.
>>>>
>>>> Any ideas?
>>>>
>>>> Thanks,
>>>> Jeff
>>>>
>>>
>>> There are some routines defined in the linux kernel for cache
>>> flushing, but under X86 they are defined to compile to nothing. I'm
>>> sure you've heard this before, but unless the CPU has some sort of
>>> problem, user space applications really should not need to flush the
>>> cache. This is not to say it can't be done, but if you want to, it
>>> means you will have to implement a flush routine in kernel space
>>> (this will of course be arch specific), export it in some way to user
>>> space and call it from there. The need to flush the cache also
>>> clearly means you can say goodbye to easy portability. Perhaps you
>>> can expand on why you need to flush the cache, and some people can
>>> propose some more portable workaround/other solutions for your problem?
>>>
>>> HTH
>>> Neil
>>
>>
>>
>> As I said, this is for a interactive language that is mostly compiled
>> and has the ability to dynamically compile to machine code,
>> new program source that is integrated into the existing program, and
>> continue executing.
>>
>> Imaine an interactive basic interpreter, that already is executing a big
>> program, and you type in a new procedure, and say to the language
>> "compile and run".
>>
>> This happens to be for a large, mature language called ICL (a strongly
>> typed garbage collected language designed for building large complex
>> systems), invented at Caltech as part of a doctoral thesis in the late
>> 1970's or so, that has around 1 millions lines of code written for it.
>> It was developed for the dec-20, ported to the vax, then the dec
>> alpha, and currently runs under windows. I am porting it to x86
>> Gnu/Linux. It was used as the backend of the first e-commerce
>> application, mosis.org, where I happened to work from 1985-1995.
>>
>> Jeff Deifik
>>
> And what specific problem are you running into that you believe will be
> solved by flushing the cache?
> Neil

Well, I am dynamically generating new code at memory addresses that
used to have old code. As the program is executing the cache may
contain the old code at the modified memory addresses. When the cache
is flushed, the memory is guaranteed to be consistent between the
cache and main memory.

This isn't something that is done very often, just after compiling new
code. If it is not done when run with microsoft windows OS's the
instructions have gotten corrupted. It is not something that is 100%
repeatable, likely due to all the other stuff accessing the cache.

thanks,
jeff deifik

jeff wrote:

> Neil Horman wrote:
>
>> jeff wrote:
>>
>>> Neil Horman wrote:
>>>
>>>> turbo wrote:
>>>>
>>>>> I have a dual processor, x86 computer running redhat 9 linux.
>>>>> I am working with a friend, on a compiler/interpreter environment
>>>>> that can dynamically generate machine code to be executed.
>>>>> In order to make the newly generated code work in the system,
>>>>> it is necessary to flush the instruction cache.
>>>>>
>>>>> On Microsoft windows, you just call
>>>>>
>>>>> FlushInstructionCache(GetCurrentProcess(), NULL, 0);
>>>>>
>>>>> I have searched, and I can't find a way to do it.
>>>>> I was able to find some old docs to do it on mips based linux.
>>>>>
>>>>> Any ideas?
>>>>>
>>>>> Thanks,
>>>>> Jeff
>>>>>
>>>>
>>>> There are some routines defined in the linux kernel for cache
>>>> flushing, but under X86 they are defined to compile to nothing. I'm
>>>> sure you've heard this before, but unless the CPU has some sort of
>>>> problem, user space applications really should not need to flush the
>>>> cache. This is not to say it can't be done, but if you want to, it
>>>> means you will have to implement a flush routine in kernel space
>>>> (this will of course be arch specific), export it in some way to
>>>> user space and call it from there. The need to flush the cache also
>>>> clearly means you can say goodbye to easy portability. Perhaps you
>>>> can expand on why you need to flush the cache, and some people can
>>>> propose some more portable workaround/other solutions for your problem?
>>>>
>>>> HTH
>>>> Neil
>>>
>>>
>>>
>>>
>>> As I said, this is for a interactive language that is mostly compiled
>>> and has the ability to dynamically compile to machine code,
>>> new program source that is integrated into the existing program, and
>>> continue executing.
>>>
>>> Imaine an interactive basic interpreter, that already is executing a big
>>> program, and you type in a new procedure, and say to the language
>>> "compile and run".
>>>
>>> This happens to be for a large, mature language called ICL (a strongly
>>> typed garbage collected language designed for building large complex
>>> systems), invented at Caltech as part of a doctoral thesis in the
>>> late 1970's or so, that has around 1 millions lines of code written
>>> for it. It was developed for the dec-20, ported to the vax, then the
>>> dec alpha, and currently runs under windows. I am porting it to x86
>>> Gnu/Linux. It was used as the backend of the first e-commerce
>>> application, mosis.org, where I happened to work from 1985-1995.
>>>
>>> Jeff Deifik
>>>
>> And what specific problem are you running into that you believe will
>> be solved by flushing the cache?
>> Neil
>
>
> Well, I am dynamically generating new code at memory addresses that
> used to have old code. As the program is executing the cache may
> contain the old code at the modified memory addresses. When the cache
> is flushed, the memory is guaranteed to be consistent between the
> cache and main memory.
>
> This isn't something that is done very often, just after compiling new
> code. If it is not done when run with microsoft windows OS's the
> instructions have gotten corrupted. It is not something that is 100%
> repeatable, likely due to all the other stuff accessing the cache.
>
> thanks,
> jeff deifik
>

I have to correct myself here. The corruption was observed running
VMS on vaxstations II, and vaxstation III's. The call to flush the cache
was added to the system at that time @1990, and was kept in place under
Microsoft Windows. I will check if the system can run without the cache
flush wunder Microsoft Windows.

Jeff Deifik

In comp.os.linux.development.system jeff wrote:
> Well, I am dynamically generating new code at memory addresses that
> used to have old code. As the program is executing the cache may
> contain the old code at the modified memory addresses. When the cache
> is flushed, the memory is guaranteed to be consistent between the
> cache and main memory.

Well, you must be running code in odd places, because the
usual code pages (.text) are read-only to permit sharing.

This is an _old_ problem, generaly known on x86 as "Self
Modifying Code". x86 has had to handle this well because
old MS-DOS programs sometimes used such tricks.

In general, it is possible and the CPU will get the right
(just written) code so long as the code being modified is _NOT_
on the current cacheline being executed. Performance of
such code is terrible, and getting worse.

There is also a Ring0 instruction `wbinvd` that will flush
the caches, but it is _horribly_ long and more used when the
CPU will power-off into battery RAM.

-- Robert