Nobody should ever need to patch the kernel!!

ROAR!!

I admit I don't know much on linux modules but there is one thing I
really hate: the need to patch the kernel. This is INSANE!

I accept that to fit a new driver into linux I have to compile the
driver source against my kernel's headers, and then do modprobe. This is
fine.

What I consider INSANE is when I need to patch and then recompile the
kernel to insert a driver or an extended the kernel functionality!?!?

WHY IS THAT NEEDED? Could't kernel developers agree on some
extensibility paradigm so that we would just need the module compile and
the modprobe?!!? I don't care if the driver would result in 5% slower,
and for many slow devices even a 5000% slower could be fine.

What, if I need two additional "drivers/features" I need to patch the
kernel twice? It is not even guaranteed that this is possible, the patch
system can refuse to apply the second patch if a certain file is already
modified by the first patch.

And hear hear, I hereby want to show that this insane method of patching
really affects security negatively:

You are a maintainer, and you have some older kernel on a machine. Now
there is a new kernel out, but you have applied say 5 patches to the old
kernel, had to fight to make that work etc... now the new kernel minor
revision comes out but the (5!) patches don't apply exactly anymore and
you badly need the features you patched... what do you do? In a real
world situation you know what you do? You DON'T upgrade the kernel!!
That's it! And in a couple of years you are vulnerable to some hack, and
unless you read security reports everyday thoroughly (LOTS of time
lost), you will soon be hacked.

Another nasty situation: mine: Under Windows I can monitor the
filesystems access with filemon (not even an installation, just
doubleclick on the program). Under linux there is no way: you need a
kernel patch to have that feature!!!! (UNBELIEVABLE!!) This would
provide useful to me for debugging in certain circumstances when some
program doesn't work, but I have a laptop with a P3-1ghz and little disk
space left. The new kernel recompile would take a lot of space and many
hours of compile time. And if I upgrade the kernel afterwards I need to
re-patch that to the new kernel. Do you think I am doing that? No way. I
do without the feature.

But this sucks.

If I am mistaken, please tell me.

And please make me understand what happens at low level, that is, why in
some cases I can do simply a compile and modprobe (e.g. nvidia driver)
and why in others this is not sufficient and I need to patch the kernel
(e.g. file monitoring)???

angryatlinux wrote:
[color=blue]
> And please make me understand what happens at low level, that is, why in
> some cases I can do simply a compile and modprobe (e.g. nvidia driver)
> and why in others this is not sufficient and I need to patch the kernel
> (e.g. file monitoring)???[/color]

Because some code can simply plug into existing kernel hooks (loadable
modules), while other functionality needs changes to code that's in the
static part of the kernel.

angryatlinux wrote:
[color=blue]
> ROAR!!
>
> I admit I don't know much on linux modules but there is one thing I
> really hate: the need to patch the kernel. This is INSANE!
>[/color]
Have you used QUILT? Use it and it will change your mind. Reading patch
is much simpler than reading modified code.
[color=blue]
> I accept that to fit a new driver into linux I have to compile the
> driver source against my kernel's headers, and then do modprobe. This is
> fine.
>
> What I consider INSANE is when I need to patch and then recompile the
> kernel to insert a driver or an extended the kernel functionality!?!?
>
> WHY IS THAT NEEDED? Could't kernel developers agree on some
> extensibility paradigm so that we would just need the module compile and
> the modprobe?!!? I don't care if the driver would result in 5% slower,
> and for many slow devices even a 5000% slower could be fine.
>[/color]
wouldn't it be mess ... different code doing samething?
[color=blue]
> What, if I need two additional "drivers/features" I need to patch the
> kernel twice? It is not even guaranteed that this is possible, the patch
> system can refuse to apply the second patch if a certain file is already
> modified by the first patch.
>
> And hear hear, I hereby want to show that this insane method of patching
> really affects security negatively:
>
> You are a maintainer, and you have some older kernel on a machine. Now
> there is a new kernel out, but you have applied say 5 patches to the old
> kernel, had to fight to make that work etc... now the new kernel minor
> revision comes out but the (5!) patches don't apply exactly anymore and
> you badly need the features you patched... what do you do? In a real
> world situation you know what you do? You DON'T upgrade the kernel!!
> That's it! And in a couple of years you are vulnerable to some hack, and
> unless you read security reports everyday thoroughly (LOTS of time
> lost), you will soon be hacked.[/color]

That happens everywhere.[color=blue]
>
> Another nasty situation: mine: Under Windows I can monitor the
> filesystems access with filemon (not even an installation, just[/color]
cool, windoze is probably for double clicking only. People who want
everything cooked in their dishes uses windoze. Its fun patching kernel
... knowing which part of kernel does what.
[color=blue]
> doubleclick on the program). Under linux there is no way: you need a
> kernel patch to have that feature!!!! (UNBELIEVABLE!!) This would
> provide useful to me for debugging in certain circumstances when some[/color]
Can you debug windoze kernel? We can debug linux kernel code...[color=blue]
> program doesn't work, but I have a laptop with a P3-1ghz and little disk
> space left. The new kernel recompile would take a lot of space and many
> hours of compile time. And if I upgrade the kernel afterwards I need to
> re-patch that to the new kernel. Do you think I am doing that? No way. I
> do without the feature.
>
> But this sucks.
>
> If I am mistaken, please tell me.
>
> And please make me understand what happens at low level, that is, why in
> some cases I can do simply a compile and modprobe (e.g. nvidia driver)
> and why in others this is not sufficient and I need to patch the kernel
> (e.g. file monitoring)???[/color]

I sympathize with you. I don't think people who live inside the kernel
could understand the frustration. My conclusion is you are either part
of the kernel distribution or you are screwed. 3rd party modules are
impractical for two reasons:

1. Modules must be compiled for every single version of the kernel they
will be used with or they are not allowed to load--this is unrealistic
to try to supply since there are infinite versions floating around out
there and constantly more. The other approach of expecting the
customers to build and install modules from source code is even more
unrealistic.

2. Modules are not allowed to see or filter things as the kernel hides
everything it can by not exporting it and provides no API's or
architecture to allow access. You are either part of the kernel or you
have no possibility of useful interaction.

It is strange to me because layered drivers have been essential in
other operating systems for the last 20 years or so. Even Dos was
actually nice in that regard and everyone had their favorite cool
tsr's. The linux model really strangles creative products.

On Wed, 11 Oct 2006 23:28:36 -0400 Rik van Riel <riel@surriel.com> wrote:
| angryatlinux wrote:
|
|> And please make me understand what happens at low level, that is, why in
|> some cases I can do simply a compile and modprobe (e.g. nvidia driver)
|> and why in others this is not sufficient and I need to patch the kernel
|> (e.g. file monitoring)???
|
| Because some code can simply plug into existing kernel hooks (loadable
| modules), while other functionality needs changes to code that's in the
| static part of the kernel.

He would probably argue that those changes in functionality are either
detrimental to the kernel, or should just already be there. Of course
there can be very good reasons such as the functionality is beyond any
value to other drivers, and wastes space being there if the driver has
no need to be used.

Obviously some things like file monitoring need unusual hooks.

FYI, I'm the kind of person that compiles my own kernel to have configured
exactly the drivers I need, monolithically, and no more than that. I even
leave out the module loading interface. While I do this for a monolithic
and tight kernel, I also recompile to add some of my own patches to the
kernel:

1. a boot parameter to let me mount a subdirectory of a partition as the
first /

2. an ioctl that lets me stuff a whole string atomically into the keyboard
input of a virtual console (must have read permission on that console
to be permitted to do so)

3. A tweak to the maximum memory allowed for tmpfs from 50% to 75% of RAM
so some of my old Sun machines can boot from a CDROM

4. Whatever I come up with next

--
|---------------------------------------/----------------------------------|
| Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
| first name lower case at ipal.net / [email]spamtrap-2006-10-12-0706@ipal.net[/email] |
|------------------------------------/-------------------------------------|

On 12 Oct 2006 01:11:41 -0700 [email]440gtx@email.com[/email] wrote:

| I sympathize with you. I don't think people who live inside the kernel
| could understand the frustration. My conclusion is you are either part
| of the kernel distribution or you are screwed. 3rd party modules are
| impractical for two reasons:
|
| 1. Modules must be compiled for every single version of the kernel they
| will be used with or they are not allowed to load--this is unrealistic
| to try to supply since there are infinite versions floating around out
| there and constantly more. The other approach of expecting the
| customers to build and install modules from source code is even more
| unrealistic.

Interfaces change sometimes. Things inside the kernel not intended to
be exposed to userland process might change for good reasons, such as
changing an integer field in a struct from 16 bits to 32 bits. All code
that works with that struct must be compiled with that change. The code
inside the kernel would all be compiled at once and always match. The
code if a driver may not.

This is one reason proprietary modules are not always a good thing.
I personally avoid them unless I have source code and can recompile
them.

It seems to me the OP may have been tasked at work creating a proprietary
driver module that is to be distributed binary only, and is frustrated by
having many different versions of the kernel to compile against (headers
of).


| 2. Modules are not allowed to see or filter things as the kernel hides
| everything it can by not exporting it and provides no API's or
| architecture to allow access. You are either part of the kernel or you
| have no possibility of useful interaction.

There are APIs for the commonly expected things. It would be impractical
to have a hook for every possible thing because then you'd end up with a
hook every 3 or 4 lines of code, and a kernel many times its size.


| It is strange to me because layered drivers have been essential in
| other operating systems for the last 20 years or so. Even Dos was
| actually nice in that regard and everyone had their favorite cool
| tsr's. The linux model really strangles creative products.

TSR was a horrible hack. It doesn't work in modern Microsoft OSes.

Why don't you describe to me a "creative product" that is being strangled
because of Linux's model for driver interfacing? Be prepared for me to
describe bad hardware design and poor engineering if you choose to do so.
"Creative" should not be "neat tricks" to use the CPU to do the hardware's
work. A classic example of a very UN-creative product is the winmodem.

--
|---------------------------------------/----------------------------------|
| Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
| first name lower case at ipal.net / [email]spamtrap-2006-10-12-0721@ipal.net[/email] |
|------------------------------------/-------------------------------------|

[email]phil-news-nospam@ipal.net[/email] wrote:[color=blue]
> He would probably argue that those changes in functionality are either
> detrimental to the kernel, or should just already be there.[/color]

Right!
[color=blue]
> Of course
> there can be very good reasons such as the functionality is beyond any
> value to other drivers, and wastes space being there if the driver has
> no need to be used.[/color]

The large amount of existing kernel patches tells you that the hooks
provided by linux kernel are way way too few!
[color=blue]
> Obviously some things like file monitoring need unusual hooks.[/color]

Unusual, really? And how come MS was wise enough to provide the required
hooks for this case? Tell me how do you do in Linux when you want to
know which process is holding a certain device "in use". Is it really a
problem that happens so rarely?


The Linux kernel does #ifdef conditional compilation for a whole lot of
things, why there aren't 2-3 macros for enabling 2-3 levels of kernel hooks?

[email]phil-news-nospam@ipal.net[/email] wrote:[color=blue]
> | It is strange to me because layered drivers have been essential in
> | other operating systems for the last 20 years or so. Even Dos was
> | actually nice in that regard and everyone had their favorite cool
> | tsr's. The linux model really strangles creative products.
>
> TSR was a horrible hack. It doesn't work in modern Microsoft OSes.
>
> Why don't you describe to me a "creative product" that is being strangled
> because of Linux's model for driver interfacing?[/color]

Oh, just look around for the most famous linux kernel patches... I don't
know many because I am a linux newbie, but anyway:

You cannot enable file monitoring or disk activity without a patch

You cannot enable shfs mounts without a patch... now this deserves a
talk on its own because it's really incredible: do you really think that
one should patch the kernel to support a new filesystem? The kernel
programmers, what were they thinking when they programmed that part,
that the number of supported filesystems by linux was not intended to raise?

You cannot enable ACPI without a patch (ok I think now it made into the
kernel, but it used to need a patch)

And others that don't come to my mind now

angryatlinux <angryatlinux@nowhere.com> writes:

[...]
[color=blue][color=green]
>> Of course
>> there can be very good reasons such as the functionality is beyond any
>> value to other drivers, and wastes space being there if the driver has
>> no need to be used.[/color]
>
> The large amount of existing kernel patches tells you that the hooks
> provided by linux kernel are way way too few![/color]

The large amount of kernel patches tells me that a large amount of
patches have been written for this kernel. It does not communicate
anything wrt the motivations why these patches have been written.
[color=blue][color=green]
>> Obviously some things like file monitoring need unusual hooks.[/color]
>
> Unusual, really?[/color]

Unusual for UNIX(*) environemnts, where a file is represented by an
i-node (simplified), which can have any number of names ('hard links')
attached to it (even zero).
[color=blue]
> And how come MS was wise enough to provide the required hooks for
> this case?[/color]

Maybe because you cannot modify the Windows-kernel?
[color=blue]
> Tell me how do you do in Linux when you want to know which process
> is holding a certain device "in use".[/color]

lsof.
[color=blue]
> Is it really a problem that happens so rarely?[/color]

Yes.
[color=blue]
> The Linux kernel does #ifdef conditional compilation for a whole lot
> of things, why there aren't 2-3 macros for enabling 2-3 levels of
> kernel hooks?[/color]

Because you can modify the source.

angryatlinux wrote:
[color=blue]
> You cannot enable file monitoring or disk activity without a patch[/color]

Sure you can. Look up "inotify".

[url]http://lwn.net/Articles/104343/[/url]
[color=blue]
> You cannot enable shfs mounts without a patch...[/color]

Based on the install instructions at
"http://shfs.sourceforge.net/install.html", you can either build and
load a kernel module, or patch and rebuild the kernel. The patch is not
required.
[color=blue]
> You cannot enable ACPI without a patch (ok I think now it made into the
> kernel, but it used to need a patch)[/color]

Current kernels certainly support ACPI. It used to need a patch because
it wasn't stable enough to be put into the mainline kernel. Vendors
shipped notoriously broken ACPI support in hardware, and were very
tight-lipped about the details.

Chris

angryatlinux wrote:[color=blue]
> [email]phil-news-nospam@ipal.net[/email] wrote:[color=green]
>> | It is strange to me because layered drivers have been essential in
>> | other operating systems for the last 20 years or so. Even Dos was
>> | actually nice in that regard and everyone had their favorite cool
>> | tsr's. The linux model really strangles creative products.
>>
>> TSR was a horrible hack. It doesn't work in modern Microsoft OSes.
>>
>> Why don't you describe to me a "creative product" that is being strangled
>> because of Linux's model for driver interfacing?[/color]
>
> Oh, just look around for the most famous linux kernel patches... I don't
> know many because I am a linux newbie, but anyway:[/color]

Then perhaps you should get a little more experience before you turn around
and make long tirades that serve only to indicate just how uninformed you are.

[color=blue]
> You cannot enable file monitoring or disk activity without a patch[/color]

Having hooks for everything is simply not feasible. There's a wide variety of
hooks available, and new ones that are deemed necessary are added all the time
-- see the 'inotify' interface in 2.6.13, for example.

[color=blue]
> You cannot enable shfs mounts without a patch... now this deserves a[/color]

Sure you can. If you bother reading the instructions, that is, instead of
whining on USENET.

[color=blue]
> talk on its own because it's really incredible: do you really think that
> one should patch the kernel to support a new filesystem? The kernel
> programmers, what were they thinking when they programmed that part,
> that the number of supported filesystems by linux was not intended to
> raise?[/color]

They did. It's called the VFS. Code that is properly modularized and uses the
VFS, should not require a kernel patch -- just a compile against your kernel
headers to crank out a module and then modprobe. Of course, the filesystem
could always use FUSE, making it even easier.

[color=blue]
> You cannot enable ACPI without a patch (ok I think now it made into the
> kernel, but it used to need a patch)[/color]

ACPI support required a patch because it was not included in mainline, until
the developers decided it was stable enough to include. That makes a lot of
sense to me. What would you have done, if it were up to you? Merge it right
away, and crash thousands of systems because of broken ACPI support?

-n

On Thu, 12 Oct 2006 16:16:29 +0200 angryatlinux <angryatlinux@nowhere.com> wrote:
| [email]phil-news-nospam@ipal.net[/email] wrote:
|> He would probably argue that those changes in functionality are either
|> detrimental to the kernel, or should just already be there.
|
| Right!
|
|> Of course
|> there can be very good reasons such as the functionality is beyond any
|> value to other drivers, and wastes space being there if the driver has
|> no need to be used.
|
| The large amount of existing kernel patches tells you that the hooks
| provided by linux kernel are way way too few!

Why clutter up the kernel with a lot of hooks?


|> Obviously some things like file monitoring need unusual hooks.
|
| Unusual, really? And how come MS was wise enough to provide the required
| hooks for this case? Tell me how do you do in Linux when you want to
| know which process is holding a certain device "in use". Is it really a
| problem that happens so rarely?

MS also makes a hideously unsafe system.

I use tools like "fuser" and "lsof" to see what is in use.


| The Linux kernel does #ifdef conditional compilation for a whole lot of
| things, why there aren't 2-3 macros for enabling 2-3 levels of kernel hooks?

Perhaps there could be. Maybe you could submit a patch with all the hooks
you think it needs, each with a #ifdef wrap so others don't have to have
all those hooks compiled in.

--
|---------------------------------------/----------------------------------|
| Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
| first name lower case at ipal.net / [email]spamtrap-2006-10-12-1849@ipal.net[/email] |
|------------------------------------/-------------------------------------|

On Thu, 12 Oct 2006 16:40:45 +0200 angryatlinux <angryatlinux@nowhere.com> wrote:
| [email]phil-news-nospam@ipal.net[/email] wrote:
|> | It is strange to me because layered drivers have been essential in
|> | other operating systems for the last 20 years or so. Even Dos was
|> | actually nice in that regard and everyone had their favorite cool
|> | tsr's. The linux model really strangles creative products.
|>
|> TSR was a horrible hack. It doesn't work in modern Microsoft OSes.
|>
|> Why don't you describe to me a "creative product" that is being strangled
|> because of Linux's model for driver interfacing?
|
| Oh, just look around for the most famous linux kernel patches... I don't
| know many because I am a linux newbie, but anyway:
|
| You cannot enable file monitoring or disk activity without a patch

Maybe because not everyone wants to have the dead code taking up space.


| You cannot enable shfs mounts without a patch... now this deserves a
| talk on its own because it's really incredible: do you really think that
| one should patch the kernel to support a new filesystem? The kernel
| programmers, what were they thinking when they programmed that part,
| that the number of supported filesystems by linux was not intended to raise?

I know there is a common interface for filesystems. What kind of patch
are you finding that is needed to add a new filesystem?


| You cannot enable ACPI without a patch (ok I think now it made into the
| kernel, but it used to need a patch)

It used to be experimental.


| And others that don't come to my mind now

You will let us know when they do.

--
|---------------------------------------/----------------------------------|
| Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
| first name lower case at ipal.net / [email]spamtrap-2006-10-12-1853@ipal.net[/email] |
|------------------------------------/-------------------------------------|

angryatlinux wrote:
[color=blue]
> [email]phil-news-nospam@ipal.net[/email] wrote:[color=green]
> > | It is strange to me because layered drivers have been essential in
> > | other operating systems for the last 20 years or so. Even Dos was
> > | actually nice in that regard and everyone had their favorite cool
> > | tsr's. The linux model really strangles creative products.[/color][/color]
ohh really? Did DOS kernel knew that TSR is there? I mean Could you
list down the TSRs with simple command like "LSMOD".[color=blue][color=green]
> >
> > TSR was a horrible hack. It doesn't work in modern Microsoft OSes.
> >
> > Why don't you describe to me a "creative product" that is being strangled
> > because of Linux's model for driver interfacing?[/color]
>
> Oh, just look around for the most famous linux kernel patches... I don't
> know many because I am a linux newbie, but anyway:
>
> You cannot enable file monitoring or disk activity without a patch
>
> You cannot enable shfs mounts without a patch... now this deserves a
> talk on its own because it's really incredible: do you really think that
> one should patch the kernel to support a new filesystem? The kernel
> programmers, what were they thinking when they programmed that part,
> that the number of supported filesystems by linux was not intended to raise?
>
> You cannot enable ACPI without a patch (ok I think now it made into the
> kernel, but it used to need a patch)
>
> And others that don't come to my mind now[/color]
Bottom line: if code is not stable it will be found as patch. If its
stable, working and useful it will be in kernel.

angryatlinux wrote:[color=blue]
> [email]phil-news-nospam@ipal.net[/email] wrote:[color=green]
> > He would probably argue that those changes in functionality are either
> > detrimental to the kernel, or should just already be there.[/color]
>
> Right!
>[color=green]
> > Of course
> > there can be very good reasons such as the functionality is beyond any
> > value to other drivers, and wastes space being there if the driver has
> > no need to be used.[/color]
>
> The large amount of existing kernel patches tells you that the hooks
> provided by linux kernel are way way too few!
>[color=green]
> > Obviously some things like file monitoring need unusual hooks.[/color]
>
> Unusual, really? And how come MS was wise enough to provide the required
> hooks for this case?[/color]

You obviously haven't been around for very log, or you would know
something about that filemon program you are using. Until quite
recently, it wasn't just available for you to click on. It was
developed by a little outfit called Sysinternals, headed by Mark
Russinovich, and available for download from their website,
[url]www.sysinternals.com[/url]. Do you know why filemon is there for you to click
on now? Because Microsoft did a little shopping bought those guys!

Yes, file monitoring requires unusual hooks. People really don't need
this on Unix. You need it on Windows because you don't know what the
heck is going on in that soup of mysterious services and DLL's. And not
to mention spyware, adware and whatever else.

On Linux, the tool you reach for most often is strace, which
instruments all of the kernel system calls that a particular process
(and, if necessary, its children) make. Not just the file-related
system calls, but everything. This tool will tell you where that
process is looking for configuration information, what kind of
networking it is doing, and so on. So strace does the job of regmon,
filemon and other kinds of "mon" all in one.

Things in Linux just don't touch files that they are not supposed to.
Moreover, if you have a daemon running as user "database", you know
it's not writing to a file owned by user "joe", which is not writable
to anyone but joe. In 17 years of using Unix and Linux, I've never had
to ask the question, WTF is accessing this file? If you're asking that
question, you're using some flavor of Windows, and you're probably
asking it because you don't know what the heck that file is and what
it's for.

There is a nice patch for Linux called kprobes. This provides a general
system for adding hooks around arbitrary machine language instructions.

If you have kprobes support in the kernel, then you can write loadable
modules which do things like monitoring file accesses. You can gain
access to the arguments of a function being called, before it is
called, and also receive its return value after it is called.
[color=blue]
> Tell me how do you do in Linux when you want to
> know which process is holding a certain device "in use". Is it really a
> problem that happens so rarely?[/color]

If it's a device that was opened using a filesystem contact point (e.g.
under /dev), then you can use the "fuser" program.
[color=blue]
> The Linux kernel does #ifdef conditional compilation for a whole lot of
> things, why there aren't 2-3 macros for enabling 2-3 levels of kernel hooks?[/color]

There obviously are these hooks, since you are able to get a patch for
them. The question is, why isn't the patch rolled into the kernel
stream? Perhaps because the kernel maintainers don't think it's a good
idea, for whatever reason.

Hooks slow code down when they are not in use (unless they are done
cleverly, like with kprobes). Every time the function is called, it has
to check the hook chain and call them out. That takes time, even if the
chain is empty, which is almost always. That hook is a debugging
feature that 99.999..% of the users out there will never use.

It's quite an acceptable model that if people want exotic features, let
them find what they are looking for on the patch market.

What about the things Microsoft /didn't/ think were a good idea to
include in their kernel? Lucky for you that the monitoring is there.
What about the user who doesn't find what he's looking for?

The user who doesn't find what he's looking for in Linux has a fighting
chance to find a patch, get someone to develop one, or do it himself if
he has the inclination and skill.

[email]phil-news-nospam@ipal.net[/email] wrote:[color=blue]
> All code that works with that struct must be compiled with that change.
> The code inside the kernel would all be compiled at once and always
> match. The code if a driver may not.[/color]

Untrue! Modules are *required* to be built against the exact kernel
they will run with. They will be just as aware and susceptible to the
most minute changes as code being recompiled in the kernel.
[color=blue]
> This is one reason proprietary modules are not always a good thing.
> I personally avoid them unless I have source code and can recompile
> them.[/color]

Designing a system where only gurus can install system level software
limits the appeal of the operating system. Computers are increasingly
used by more general types who don't want to know what a kernel is
these days.
[color=blue]
> There are APIs for the commonly expected things. It would be impractical
> to have a hook for every possible thing because then you'd end up with a
> hook every 3 or 4 lines of code, and a kernel many times its size.[/color]

Just offer simple methods to access existing things in architected ways
each with its own versioning. Leave the performance path and internals
alone. Just about everything is already there anyway like kobjects.
Insulate modules, don't kill them on every new build.
[color=blue]
> TSR was a horrible hack[/color]

You must admit they could be written to work on every single version of
dos and they added terrific features that everyone used. Sidekick, disk
caches, key recalls, mouse, cdrom, >540MB hard drives, you name it, it
could and was done. Dos would have been very boring without them, hmm.
[color=blue]
> It doesn't work in modern Microsoft OSes.[/color]

I beg to differ. Windows allows installing 3rd party drivers just as
easily as installing applications and you can easily support all the
important windows versions with one package. There is some very cool
stuff out there that can't be ported to linux because of the kernel
nightmare situation.
[color=blue]
> Why don't you describe to me a "creative product" that is being
> strangled because of Linux's model for driver interfacing?[/color]

This I believe is the fundamental problem. I notice a lot of "if I
can't think of a reason, there must not be any and therefore I will
lock the door to the possibliity". This is fine if you are designing
something for yourself, but there are unlimited numbers of creative
people out there who have ideas you will never dream of and create new
classes of products that need non traditional device drivers you never
thought about.
[color=blue]
> "Creative" should not be "neat tricks" to use the CPU to do the hardware's
> work. A classic example of a very UN-creative product is the winmodem.[/color]

I don't know the hardware specifics of winmodem in particular, but the
key is if you move intelligence out of the hardware, you need to have
an extra intelligent person writing the driver to make it work right.
For example, IDE and all its protocols and DMA modes is much more
primitive to control than fancy SCSI hardware, but it works great if
you write a good driver for it. But put some random joe on it and it
will be full of problems--perhaps what happened to winmodem? But sure,
some hardware can be fundamentally flawed (polling, static delays,
etc). Fancy hardware is not always good because it tends to cost more.
That's why IDE dominated SCSI and USB dominated firewire. Cost wins.

Nikolaos D. Bougalis wrote:[color=blue][color=green]
>> You cannot enable ACPI without a patch (ok I think now it made into
>> the kernel, but it used to need a patch)[/color]
>
> ACPI support required a patch because it was not included in
> mainline, until the developers decided it was stable enough to include.
> That makes a lot of sense to me. What would you have done, if it were up
> to you? Merge it right away, and crash thousands of systems because of
> broken ACPI support?[/color]

Various people replied this thing about ACPI. These people haven't
understood what I'm writing! I'm not complaining because there are
things available as patch vs directly in the kernel, I'm complaining
because there are things available in patches vs in modules!

ACPI is one example, Kprobes is another one... ok I was mistaken with
the file monitoring and shfs.

Patches are a hell to maintain, see my first post for why, useless to
repeat here.

On Fri, 13 Oct 2006 14:30:35 +0200 angryatlinux <angryatlinux@nowhere.com> wrote:
| Nikolaos D. Bougalis wrote:
|>> You cannot enable ACPI without a patch (ok I think now it made into
|>> the kernel, but it used to need a patch)
|>
|> ACPI support required a patch because it was not included in
|> mainline, until the developers decided it was stable enough to include.
|> That makes a lot of sense to me. What would you have done, if it were up
|> to you? Merge it right away, and crash thousands of systems because of
|> broken ACPI support?
|
| Various people replied this thing about ACPI. These people haven't
| understood what I'm writing! I'm not complaining because there are
| things available as patch vs directly in the kernel, I'm complaining
| because there are things available in patches vs in modules!

No, you were complaining that some things which operate as modules also
need a patch to the kernel as well. Many modules you might choose to
configure in when you compile a kernel also include the needed hooks.


| ACPI is one example, Kprobes is another one... ok I was mistaken with
| the file monitoring and shfs.
|
| Patches are a hell to maintain, see my first post for why, useless to
| repeat here.

External patches do have that problem. But if the patch were rolled into
the mainline kernel source tree, then it would be just that much more for
teh kernel maintainers to do, too, in addition to a larger kernel. That's
not worth doing unless enough people want it.

Microsoft does seem to put a lot more in Windows. Every business that can
afford to pay them off to include something can probably get it in there.
They we all pay for having a larger kernel, more bugs, and mroe security
issues, when we're not using something better.

--
|---------------------------------------/----------------------------------|
| Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
| first name lower case at ipal.net / [email]spamtrap-2006-10-13-1319@ipal.net[/email] |
|------------------------------------/-------------------------------------|

On 13 Oct 2006 01:30:18 -0700 [email]440gtx@email.com[/email] wrote:
| [email]phil-news-nospam@ipal.net[/email] wrote:
|> All code that works with that struct must be compiled with that change.
|> The code inside the kernel would all be compiled at once and always
|> match. The code if a driver may not.
|
| Untrue! Modules are *required* to be built against the exact kernel
| they will run with. They will be just as aware and susceptible to the
| most minute changes as code being recompiled in the kernel.

I've run mismatched modules before. They even ran OK.


|> This is one reason proprietary modules are not always a good thing.
|> I personally avoid them unless I have source code and can recompile
|> them.
|
| Designing a system where only gurus can install system level software
| limits the appeal of the operating system. Computers are increasingly
| used by more general types who don't want to know what a kernel is
| these days.

Well at least I _can_ install system level software in Linux. You can't
even do that at all in Windows.


|> There are APIs for the commonly expected things. It would be impractical
|> to have a hook for every possible thing because then you'd end up with a
|> hook every 3 or 4 lines of code, and a kernel many times its size.
|
| Just offer simple methods to access existing things in architected ways
| each with its own versioning. Leave the performance path and internals
| alone. Just about everything is already there anyway like kobjects.
| Insulate modules, don't kill them on every new build.

Actually what we need are fewer modules.


|> TSR was a horrible hack
|
| You must admit they could be written to work on every single version of
| dos and they added terrific features that everyone used. Sidekick, disk
| caches, key recalls, mouse, cdrom, >540MB hard drives, you name it, it
| could and was done. Dos would have been very boring without them, hmm.

I never used any of them.


|> It doesn't work in modern Microsoft OSes.
|
| I beg to differ. Windows allows installing 3rd party drivers just as
| easily as installing applications and you can easily support all the
| important windows versions with one package. There is some very cool
| stuff out there that can't be ported to linux because of the kernel
| nightmare situation.

Only drivers from companies that pay through the nose to Microsoft for
the licensed development products and key signings.


|> Why don't you describe to me a "creative product" that is being
|> strangled because of Linux's model for driver interfacing?
|
| This I believe is the fundamental problem. I notice a lot of "if I
| can't think of a reason, there must not be any and therefore I will
| lock the door to the possibliity". This is fine if you are designing
| something for yourself, but there are unlimited numbers of creative
| people out there who have ideas you will never dream of and create new
| classes of products that need non traditional device drivers you never
| thought about.

You didn't want to describe a real example because whatever it is, I
could describe why it doesn't really need a driver in the first place.

So until you do come up with some examples, even hypothetical ones,
I'm just chalking up the above as "blowing more smoke".


|> "Creative" should not be "neat tricks" to use the CPU to do the hardware's
|> work. A classic example of a very UN-creative product is the winmodem.
|
| I don't know the hardware specifics of winmodem in particular, but the
| key is if you move intelligence out of the hardware, you need to have
| an extra intelligent person writing the driver to make it work right.
| For example, IDE and all its protocols and DMA modes is much more
| primitive to control than fancy SCSI hardware, but it works great if
| you write a good driver for it. But put some random joe on it and it
| will be full of problems--perhaps what happened to winmodem? But sure,
| some hardware can be fundamentally flawed (polling, static delays,
| etc). Fancy hardware is not always good because it tends to cost more.
| That's why IDE dominated SCSI and USB dominated firewire. Cost wins.

It's the wrong way to go about it. There are numerous different operating
systems. But the manufacturer clearly didn't want to make a driver for all
of them. The reason had nothing to do with kernel versions in Linux. It
had everything to do with the fact that they were making a cheap product
and didn't want to put forth any effort to make a driver for any other OS.

The correct design would be to NOT move the intelligence out to the CPU.
It should be left in the hardware itself. The interface from the software
running the CPU and the hardware should be basic and simple and common to
all hardware of the same class (e.g. every modem should use the very same
interface). Then if you want to make a hardware product that runs faster
or processed bits better under noisy conditions or whatever, put all that
in the hardware where it's safe. Then any OS can use it.

--
|---------------------------------------/----------------------------------|
| Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
| first name lower case at ipal.net / [email]spamtrap-2006-10-13-1325@ipal.net[/email] |
|------------------------------------/-------------------------------------|

> I've run mismatched modules before. They even ran OK.

There are distributions configured to where using insmod with the force
option is ignored. Requiring a blessed license type is also coming into
play for what it is worth.
[color=blue]
> Only drivers from companies that pay through the nose to Microsoft for
> the licensed development products and key signings.[/color]

FYI, the DDK has been free for about 5 years. Heck, there is even a
free version of visual studio these days. Logo for applications and
signing of drivers remains optional to this day. However Vista will
have strict signing requirements for x64 drivers. Microsoft believes
this will make it more stable and secure, we'll see. But up to now it's
been easy to ship drivers, just press the build button and you've got a
binary you can give to customers with no required costs or other
overhead.
[color=blue]
> So until you do come up with some examples, even hypothetical ones,
> I'm just chalking up the above as "blowing more smoke".[/color]

Ok, let's get to a real example. How about tracing and bus analysis
tools? Is that specific enough? There are many productivity products on
Windows that log software packets and hardware protocol that are
invaluable to firmware and device guys. For linux, people are limited
to attaching black boxes to their buses which come with significant
tradeoffs, many of them negative. Think of strace, but for low kernel
level data structures like block device requests or SCSI protocol. It
is not clear to me a module has any way to attach so that it could
capture these type things. On the other hand, roll back 10 years and
even windows and os/2 already had the concept of filter drivers.
[color=blue]
> It's the wrong way to go about it. There are numerous different operating
> systems. But the manufacturer clearly didn't want to make a driver for all
> of them. The reason had nothing to do with kernel versions in Linux.[/color]

Ah, I see you are referring to the proprietary nature of the hardware
interface and concur that's typically a bad idea in the long run. As a
side note, as someone who accidentally owned a WinModem for a time, it
was about the worst hardware experience I have had. The driver had the
most horrible side effects on the system. It was very poorly written
and the developer should have been shot.