Tool for - Userspace to Kernelspace - Linux

This is a discussion on Tool for - Userspace to Kernelspace - Linux ; Hi, Kindly tell some free tool that would help in transferring an application from Userspace to Kernelspace. I tried google, but was unable to get it. Kindly share some links / views / suggestions / pdfs. Thx in advans, Karthik ...

+ Reply to Thread
Results 1 to 12 of 12

Thread: Tool for - Userspace to Kernelspace

  1. Tool for - Userspace to Kernelspace

    Hi,

    Kindly tell some free tool that would help in transferring an
    application from Userspace to Kernelspace.

    I tried google, but was unable to get it.

    Kindly share some links / views / suggestions / pdfs.

    Thx in advans,
    Karthik Balaguru


  2. Re: Tool for - Userspace to Kernelspace

    karthikbg wrote:
    > Hi,
    >
    > Kindly tell some free tool that would help in transferring an
    > application from Userspace to Kernelspace.
    >
    > I tried google, but was unable to get it.
    >
    > Kindly share some links / views / suggestions / pdfs.


    Why on earth would you want to do that?
    I cannot think of any good reason to do so.
    And I can think of numerous reasons not to do so, e.g. security reasons,
    or the fact that kernel code is not paged/swapped.

    Or just, maybe, I have not fully understood what you want to do.
    --
    Josef Möllers (Pinguinpfleger bei FSC)
    If failure had no penalty success would not be a prize
    -- T. Pratchett


  3. Re: Tool for - Userspace to Kernelspace

    Josef Moellers writes:
    > karthikbg wrote:
    >> Hi,
    >> Kindly tell some free tool that would help in transferring an
    >> application from Userspace to Kernelspace.
    >> I tried google, but was unable to get it.
    >> Kindly share some links / views / suggestions / pdfs.

    >
    > Why on earth would you want to do that?
    > I cannot think of any good reason to do so.


    The 'usual suspects' explanation would be 'code performs badly in
    userspace, nobody has any idea, why, but not running inside the kernel
    is certainly associated with more overhead' ....

  4. Re: Tool for - Userspace to Kernelspace

    >>>Kindly tell some free tool that would help in transferring an
    >>>application from Userspace to Kernelspace.


    >>Why on earth would you want to do that?
    >>I cannot think of any good reason to do so.


    > The 'usual suspects' explanation would be 'code performs badly in
    > userspace, nobody has any idea, why, but not running inside the kernel
    > is certainly associated with more overhead' ....


    Measure the number, distribution, and frequency of system calls from user
    space to kernel space. Only a program such as 'find', 'make', or a caller
    of nftw() can benefit substantially from removing the overhead of system
    calls; but it would be a gigantic risk of system instability and insecurity
    to put such a program inside the kernel. In many threaded programs
    the most important calls are gettimeofday() and/or gettid(). gettid()
    has been optimized by glibc, and gettimeofday() has been optimized
    by the kernel.

    Another possibility is to improve the efficiency of memory caches
    by avoiding bad page coloring. Linux selects physical page frames
    randomly on demand. Instead, for uniformly best cache efficiency
    in a particular process, physical frames should be selected to spread
    the cache collisions evenly. This matters in current processors,
    which have many cache lines (>=4K), few cache lines per page (<=128),
    and small associativity (<=8). On average, random frame selection
    is not bad. However, I have seen execution times for the same
    user-mode cache-bound process vary by 20% or more due to collision
    rates that were adversely affected by poor choice of page frames.
    The best "blind" policy (one that knows nothing about the particular
    process) is physically sequential frames for sequential virtual pages.
    This can be approximated by using HUGETLBFS. A program that
    intensively processes large arrays probably should investigate
    this approach.

    --

  5. Re: Tool for - Userspace to Kernelspace

    On Mon, 6 Nov 2006, John Reiser wrote:

    >>>> Kindly tell some free tool that would help in transferring an
    >>>> application from Userspace to Kernelspace.

    >
    >>> Why on earth would you want to do that?
    >>> I cannot think of any good reason to do so.

    >
    >> The 'usual suspects' explanation would be 'code performs badly in
    >> userspace, nobody has any idea, why, but not running inside the kernel
    >> is certainly associated with more overhead' ....

    >
    > Measure the number, distribution, and frequency of system calls from user
    > space to kernel space. Only a program such as 'find', 'make', or a caller
    > of nftw() can benefit substantially from removing the overhead of system
    > calls; but it would be a gigantic risk of system instability and insecurity
    > to put such a program inside the kernel. In many threaded programs
    > the most important calls are gettimeofday() and/or gettid(). gettid()
    > has been optimized by glibc, and gettimeofday() has been optimized
    > by the kernel.
    >
    > Another possibility is to improve the efficiency of memory caches
    > by avoiding bad page coloring. Linux selects physical page frames
    > randomly on demand. Instead, for uniformly best cache efficiency
    > in a particular process, physical frames should be selected to spread
    > the cache collisions evenly. This matters in current processors,
    > which have many cache lines (>=4K), few cache lines per page (<=128),
    > and small associativity (<=8). On average, random frame selection
    > is not bad. However, I have seen execution times for the same
    > user-mode cache-bound process vary by 20% or more due to collision
    > rates that were adversely affected by poor choice of page frames.
    > The best "blind" policy (one that knows nothing about the particular
    > process) is physically sequential frames for sequential virtual pages.
    > This can be approximated by using HUGETLBFS. A program that
    > intensively processes large arrays probably should investigate
    > this approach.
    >
    >


    I think is a neat idea to merge address spaces esp with the
    advent of VMs. You can have kernel specially tailored for your
    app or set of apps. Why pay the extra cost in switching modes!

    There are obvious ramifications like some of you have mentioned.
    But it is not that spine-chilling idea to try
    S
    ************************************************** ***********************
    "I've never understood how God could expect his creatures
    to pick the one true religion by faith - it strikes me as a
    sloppy way to run a universe."
    Robert A. Heinlein
    STRANGER IN A STRANGE LAND
    ************************************************** ***********************

  6. Re: Tool for - Userspace to Kernelspace

    John Reiser writes:
    >>>>Kindly tell some free tool that would help in transferring an
    >>>>application from Userspace to Kernelspace.

    >
    >>>Why on earth would you want to do that?
    >>>I cannot think of any good reason to do so.

    >
    >> The 'usual suspects' explanation would be 'code performs badly in
    >> userspace, nobody has any idea, why, but not running inside the kernel
    >> is certainly associated with more overhead' ....

    >
    > Measure the number, distribution, and frequency of system calls from user
    > space to kernel space.


    I did not intend to report my opinion regarding this subject, but one
    that I have already encountered among others, in a slightly parodized
    way. But anyway ...

    It is somewhat unclear what the 'distribution' of syscalls is supposed
    to be. Assuming I know the frequency, the number should be frequency *
    time. But all of this is a little too general ot realistically
    describe 'system call overhead'.

    > Only a program such as 'find', 'make', or a caller
    > of nftw() can benefit substantially from removing the overhead of system
    > calls;


    Any program that does mainly I/O-processing (eg all types of network
    servers, network clients, proxies, media applications) would, 'naively
    seen', benefit from running inside the kernel, because input data would
    not need to be copied from kernel to userspace, output data not from
    userspace to kernel and there wouldn't be the need to context-switch
    between user and kernel mode before processing of something could
    start and forwarding of the outputs of processing could take place
    (for instance, I know of at least on US-based company that offers
    kernel-level WWW traffic inspection and content-filtering). These
    benefits would even extend to multi-threaded applications, because the
    Linux-kernel support multiple threads of execution in kernel space.

    A reminder: I am not talking about my opinion wrt to that but just
    trying to (shortly) sum up why someone would want to run things
    in-kernel that could also be implemented using ordinary
    userspace-applications.

    > but it would be a gigantic risk of system instability and
    > insecurity to put such a program inside the kernel.


    It is not unusual practice for all kinds of 'embedded' applications to
    run all code inside the kernel. And kernel programming isn't
    significantly more difficult than userspace programming, assuming the
    person doing it knows what he or she is doing (that there are so many
    people programming computers that decidedly don't, but instead cling to
    all kinds of fantastic beliefs about the machine and how it works [at
    the software level] is a different issue).

    > In many threaded programs the most important calls are gettimeofday() and/or
    > gettid(). gettid() has been optimized by glibc, and gettimeofday()
    > has been optimized by the kernel.
    >
    > Another possibility is to improve the efficiency of memory caches
    > by avoiding bad page coloring.


    Many programs (even many multi-threaded programs) are 'controllers'
    ('drivers') rather than 'calculators' (you appear to refer to number
    crunching applications).

  7. Re: Tool for - Userspace to Kernelspace

    >>>The 'usual suspects' explanation would be 'code performs badly in
    >>>userspace, nobody has any idea, why, but not running inside the kernel
    >>>is certainly associated with more overhead' ....


    >>Measure the number, distribution, and frequency of system calls from user
    >>space to kernel space.


    > It is somewhat unclear what the 'distribution' of syscalls is supposed
    > to be. Assuming I know the frequency, the number should be frequency *
    > time. But all of this is a little too general ot realistically
    > describe 'system call overhead'.


    'Distribution' means the shape of the histogram when sorted by frequency.
    Are there just 1 or 2 system calls whose usage is several times that of
    any other, etc?

    >>Only a program such as 'find', 'make', or a caller
    >>of nftw() can benefit substantially from removing the overhead of system
    >>calls;

    >
    >
    > Any program that does mainly I/O-processing (eg all types of network
    > servers, network clients, proxies, media applications) would, 'naively
    > seen', benefit from running inside the kernel, because input data would
    > not need to be copied from kernel to userspace,


    The Linux kernel is far beyond 'naive'. mmap(), zero-copy data paths
    for network transmission, sendfile(), and splice() already subsume
    and/or cover a large portion of such copying.

    output data not from
    > userspace to kernel and there wouldn't be the need to context-switch
    > between user and kernel mode before processing of something could
    > start and forwarding of the outputs of processing could take place
    > (for instance, I know of at least on US-based company that offers
    > kernel-level WWW traffic inspection and content-filtering). These
    > benefits would even extend to multi-threaded applications, because the
    > Linux-kernel support multiple threads of execution in kernel space.


    Removing the safety harnesses of any physical activity reduces the
    overhead due to size, weight, and mechanical interferences.

    >
    > A reminder: I am not talking about my opinion wrt to that but just
    > trying to (shortly) sum up why someone would want to run things
    > in-kernel that could also be implemented using ordinary
    > userspace-applications.
    >
    >
    >>but it would be a gigantic risk of system instability and
    >>insecurity to put such a program inside the kernel.

    >
    >
    > It is not unusual practice for all kinds of 'embedded' applications to
    > run all code inside the kernel. ...


    An essential property of 'embedded' is that the application universe
    is tightly bounded and controlled. This is so important that any
    discussion must _begin_ with this qualification, not add it later.

    --

  8. Re: Tool for - Userspace to Kernelspace

    On Mon, 06 Nov 2006 09:34:28 +0100 Josef Moellers wrote:

    | Why on earth would you want to do that?
    | I cannot think of any good reason to do so.

    Actually, I have contemplated putting an authoritative-only DNS server
    in kernel space, probably right in the network stack or even at the
    interface driver.


    | And I can think of numerous reasons not to do so, e.g. security reasons,
    | or the fact that kernel code is not paged/swapped.

    Not being paged/swapped can be good for some things. With a DNS server,
    certainly security will be an issue. Very careful programming is a must.

    Such a DNS server would likely need to have all the data pre-loaded. But
    one approach I could take is to use the kernel space as a shortcut. If
    the data is present when the datagram arrives, shoot the answer back right
    then. If not, then let it pass on to the userland server which would be
    the way to trigger loading it. Another way would be as an authoritative
    cache where the queries get forwarded if they miss, either to userland or
    another server. Then watch for answers coming back and cache those.

    --
    |---------------------------------------/----------------------------------|
    | Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
    | first name lower case at ipal.net / spamtrap-2006-11-06-1500@ipal.net |
    |------------------------------------/-------------------------------------|

  9. Re: Tool for - Userspace to Kernelspace

    John Reiser writes:

    [...]

    >> Any program that does mainly I/O-processing (eg all types of network
    >> servers, network clients, proxies, media applications) would, 'naively
    >> seen', benefit from running inside the kernel, because input data would
    >> not need to be copied from kernel to userspace,

    >
    > The Linux kernel is far beyond 'naive'.


    The Linux kernel is a program and the adjective 'naive', used with the
    meaning I used it above, ie to describe a human perspective, cannot be
    applied to it. It could have told you that I do not entirely agree
    with the viewpoint I have described.

    > mmap(), zero-copy data paths for network transmission, sendfile(),
    > and splice() already subsume and/or cover a large portion of such
    > copying.


    Would you be so kind to stop throwing random terms at me without a
    reason?

    > output data not from> userspace to kernel and there wouldn't be the
    > need to context-switch between user and kernel mode before
    > processing of something could start and forwarding of the outputs
    > of processing could take place (for instance, I know of at least on
    > US-based company that offers kernel-level WWW traffic inspection
    > and content-filtering). These benefits would even extend to
    > multi-threaded applications, because the Linux-kernel support
    > multiple threads of execution in kernel space.
    >
    > Removing the safety harnesses of any physical activity reduces the
    > overhead due to size, weight, and mechanical interferences.


    And please apply the same to random flawed analogies.

  10. Re: Tool for - Userspace to Kernelspace

    Hi,

    Mine is a Flash Memory driver with lot of handles in userspace.

    I came across something interesting related to this via google.
    It is KML . I think it would be useful .

    The Interesting thing is 'bash' gets executed from 'kernel mode'


    For example, the following commands execute bash in kernel mode.:
    % cp /bin/bash /trusted/bin && /trusted/bin/bash

    KML is a modified Linux kernel that executes user processes in kernel
    mode, which then are called kernel-mode user processes. Kernel-mode
    user processes can interact with the kernel directly. Therefore, the
    overhead of system calls can be eliminated.
    KML is provided as a patch to the source of the original Linux kernel,
    so you need to build the kernel from the source.

    To use KML, apply the patch and enable Kernel Mode Linux when you
    configure your kernel. Build and install the kernel, and then reboot.
    The KML patch is available from www.yl.is.s.u-tokyo.ac.jp/~tosh/kml.

    what do you think about this ?

    Regards,
    Karthik Balaguru


  11. Re: Tool for - Userspace to Kernelspace

    Rainer Weikusat wrote:
    > Any program that does mainly I/O-processing (eg all types of network
    > servers, network clients, proxies, media applications) would, 'naively
    > seen', benefit from running inside the kernel, because input data would
    > not need to be copied from kernel to userspace, output data not from


    The bulk copying of data cannot be eliminated simply by moving a
    program to the kernel. Its interaction with the OS must be redesigned
    to use non-copying interfaces. But this can be done without moving it
    to the kernel, if those interfaces are available. In other words, the
    data copying issue is orthogonal to the question of privilege mode.

    The other benefits that you noted, like eliminating the mode switch
    overhead, are valid.

    I have something to add also: in my experience, it's easier to develop
    robust, bullet-proof kernel code than user space code with the same
    attributes.

    When you code something in the kernel, it's a lot closer to being
    absolute.

    You don't have to worry about stupid questions like "what will happen
    to this process-shared state if a SIGKILL were to happen right now".
    But you do have to worry about questions like "if this gets into a
    loop, I won't be able to SIGKILL". If you're smart, you know which one
    is easier.


  12. Re: Tool for - Userspace to Kernelspace

    karthikbg wrote in part:
    > Mine is a Flash Memory driver with lot of handles in userspace.


    Interfaced across USB? You might want to look into some of the
    "zero copy" routines used by gigabit network drivers. AFAIK,
    the basic idea is to drop bits directly into userspace memory.

    -- Robert


+ Reply to Thread