[PATCH] procfs: provide slub's /proc/slabinfo - Kernel

This is a discussion on [PATCH] procfs: provide slub's /proc/slabinfo - Kernel ; SLUB's new slabinfo isn't there: it looks as if a last minute change to Pekka's patch left it dependent on CONFIG_SLAB at the procfs end: allow for CONFIG_SLUB too. Signed-off-by: Hugh Dickins --- To minimize ifdeffery, this leaves it with ...

+ Reply to Thread
Page 1 of 4 1 2 3 ... LastLast
Results 1 to 20 of 68

Thread: [PATCH] procfs: provide slub's /proc/slabinfo

  1. [PATCH] procfs: provide slub's /proc/slabinfo

    SLUB's new slabinfo isn't there: it looks as if a last minute change
    to Pekka's patch left it dependent on CONFIG_SLAB at the procfs end:
    allow for CONFIG_SLUB too.

    Signed-off-by: Hugh Dickins
    ---
    To minimize ifdeffery, this leaves it with S_IWUSR though unwritable:
    I'm assuming that's acceptable.

    fs/proc/proc_misc.c | 7 +++++--
    1 file changed, 5 insertions(+), 2 deletions(-)

    --- 2.6.24-rc6-git/fs/proc/proc_misc.c 2007-10-20 08:04:13.000000000 +0100
    +++ linux/fs/proc/proc_misc.c 2008-01-02 17:55:57.000000000 +0000
    @@ -410,15 +410,18 @@ static const struct file_operations proc
    };
    #endif

    -#ifdef CONFIG_SLAB
    +#if defined(CONFIG_SLAB) || defined(CONFIG_SLUB)
    static int slabinfo_open(struct inode *inode, struct file *file)
    {
    return seq_open(file, &slabinfo_op);
    }
    +
    static const struct file_operations proc_slabinfo_operations = {
    .open = slabinfo_open,
    .read = seq_read,
    +#ifdef CONFIG_SLAB
    .write = slabinfo_write,
    +#endif
    .llseek = seq_lseek,
    .release = seq_release,
    };
    @@ -728,7 +731,7 @@ void __init proc_misc_init(void)
    #endif
    create_seq_entry("stat", 0, &proc_stat_operations);
    create_seq_entry("interrupts", 0, &proc_interrupts_operations);
    -#ifdef CONFIG_SLAB
    +#if defined(CONFIG_SLAB) || defined(CONFIG_SLUB)
    create_seq_entry("slabinfo",S_IWUSR|S_IRUGO,&proc_slabinfo_operations);
    #ifdef CONFIG_DEBUG_SLAB_LEAK
    create_seq_entry("slab_allocators", 0 ,&proc_slabstats_operations);
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  2. Re: [PATCH] procfs: provide slub's /proc/slabinfo

    On Wed, 2 Jan 2008, Hugh Dickins wrote:

    > SLUB's new slabinfo isn't there: it looks as if a last minute change
    > to Pekka's patch left it dependent on CONFIG_SLAB at the procfs end:
    > allow for CONFIG_SLUB too.
    >
    > Signed-off-by: Hugh Dickins


    I just saw the patch in Linus tree and wondered how this is going to work
    without the piece that you are providing here.

    Acked-by: Christoph Lameter

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  3. Re: [PATCH] procfs: provide slub's /proc/slabinfo

    Hi,

    On Jan 2, 2008 8:43 PM, Hugh Dickins wrote:
    > SLUB's new slabinfo isn't there: it looks as if a last minute change
    > to Pekka's patch left it dependent on CONFIG_SLAB at the procfs end:
    > allow for CONFIG_SLUB too.
    >
    > Signed-off-by: Hugh Dickins
    > ---
    > To minimize ifdeffery, this leaves it with S_IWUSR though unwritable:
    > I'm assuming that's acceptable.


    I already sent the remaining bits to Linus but this looks much
    cleaner. Thanks Hugh!

    Acked-by: Pekka Enberg
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  4. Re: [PATCH] procfs: provide slub's /proc/slabinfo



    On Wed, 2 Jan 2008, Pekka Enberg wrote:
    >
    > I already sent the remaining bits to Linus but this looks much
    > cleaner. Thanks Hugh!
    >
    > Acked-by: Pekka Enberg


    Actually, I'd much rather just do this instead (on top of your patch)

    It just creates a new CONFIG_SLABINFO that automatically has the right
    dependencies (ie depends on PROC being on, and either SLAB or SLUB), and
    then both SLAB and SLUB just have the exact same interfaces.

    Which means that SLOB could also trivially implement the same thing, with
    no new #ifdef'fery or other crud.

    It's totally untested, but looks fairly obvious.

    Linus

    ---
    fs/proc/proc_misc.c | 21 ++-------------------
    include/linux/slab.h | 5 +++++
    include/linux/slab_def.h | 3 ---
    include/linux/slub_def.h | 2 --
    init/Kconfig | 6 ++++++
    mm/slab.c | 2 +-
    mm/slub.c | 11 +++++++++--
    7 files changed, 23 insertions(+), 27 deletions(-)

    diff --git a/fs/proc/proc_misc.c b/fs/proc/proc_misc.c
    index a11968b..3462bfd 100644
    --- a/fs/proc/proc_misc.c
    +++ b/fs/proc/proc_misc.c
    @@ -410,7 +410,7 @@ static const struct file_operations proc_modules_operations = {
    };
    #endif

    -#ifdef CONFIG_SLAB
    +#ifdef CONFIG_SLABINFO
    static int slabinfo_open(struct inode *inode, struct file *file)
    {
    return seq_open(file, &slabinfo_op);
    @@ -451,20 +451,6 @@ static const struct file_operations proc_slabstats_operations = {
    #endif
    #endif

    -#ifdef CONFIG_SLUB
    -static int slabinfo_open(struct inode *inode, struct file *file)
    -{
    - return seq_open(file, &slabinfo_op);
    -}
    -
    -static const struct file_operations proc_slabinfo_operations = {
    - .open = slabinfo_open,
    - .read = seq_read,
    - .llseek = seq_lseek,
    - .release = seq_release,
    -};
    -#endif
    -
    static int show_stat(struct seq_file *p, void *v)
    {
    int i;
    @@ -742,15 +728,12 @@ void __init proc_misc_init(void)
    #endif
    create_seq_entry("stat", 0, &proc_stat_operations);
    create_seq_entry("interrupts", 0, &proc_interrupts_operations);
    -#ifdef CONFIG_SLAB
    +#ifdef CONFIG_SLABINFO
    create_seq_entry("slabinfo",S_IWUSR|S_IRUGO,&proc_slabinfo_operations);
    #ifdef CONFIG_DEBUG_SLAB_LEAK
    create_seq_entry("slab_allocators", 0 ,&proc_slabstats_operations);
    #endif
    #endif
    -#ifdef CONFIG_SLUB
    - create_seq_entry("slabinfo", S_IWUSR|S_IRUGO, &proc_slabinfo_operations);
    -#endif
    create_seq_entry("buddyinfo",S_IRUGO, &fragmentation_file_operations);
    create_seq_entry("pagetypeinfo", S_IRUGO, &pagetypeinfo_file_ops);
    create_seq_entry("vmstat",S_IRUGO, &proc_vmstat_file_operations);
    diff --git a/include/linux/slab.h b/include/linux/slab.h
    index f3a8eec..f62caaa 100644
    --- a/include/linux/slab.h
    +++ b/include/linux/slab.h
    @@ -271,5 +271,10 @@ static inline void *kzalloc(size_t size, gfp_t flags)
    return kmalloc(size, flags | __GFP_ZERO);
    }

    +#ifdef CONFIG_SLABINFO
    +extern const struct seq_operations slabinfo_op;
    +ssize_t slabinfo_write(struct file *, const char __user *, size_t, loff_t *);
    +#endif
    +
    #endif /* __KERNEL__ */
    #endif /* _LINUX_SLAB_H */
    diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h
    index 32bdc2f..fcc4809 100644
    --- a/include/linux/slab_def.h
    +++ b/include/linux/slab_def.h
    @@ -95,7 +95,4 @@ found:

    #endif /* CONFIG_NUMA */

    -extern const struct seq_operations slabinfo_op;
    -ssize_t slabinfo_write(struct file *, const char __user *, size_t, loff_t *);
    -
    #endif /* _LINUX_SLAB_DEF_H */
    diff --git a/include/linux/slub_def.h b/include/linux/slub_def.h
    index b7d9408..40801e7 100644
    --- a/include/linux/slub_def.h
    +++ b/include/linux/slub_def.h
    @@ -200,6 +200,4 @@ static __always_inline void *kmalloc_node(size_t size, gfp_t flags, int node)
    }
    #endif

    -extern const struct seq_operations slabinfo_op;
    -
    #endif /* _LINUX_SLUB_DEF_H */
    diff --git a/init/Kconfig b/init/Kconfig
    index 404bbf3..b9d11a8 100644
    --- a/init/Kconfig
    +++ b/init/Kconfig
    @@ -658,6 +658,12 @@ endchoice

    endmenu # General setup

    +config SLABINFO
    + bool
    + depends on PROC_FS
    + depends on SLAB || SLUB
    + default y
    +
    config RT_MUTEXES
    boolean
    select PLIST
    diff --git a/mm/slab.c b/mm/slab.c
    index 2e338a5..aebb9f6 100644
    --- a/mm/slab.c
    +++ b/mm/slab.c
    @@ -4105,7 +4105,7 @@ out:
    schedule_delayed_work(work, round_jiffies_relative(REAPTIMEOUT_CPUC));
    }

    -#ifdef CONFIG_PROC_FS
    +#ifdef CONFIG_SLABINFO

    static void print_slabinfo_header(struct seq_file *m)
    {
    diff --git a/mm/slub.c b/mm/slub.c
    index 903dabd..474945e 100644
    --- a/mm/slub.c
    +++ b/mm/slub.c
    @@ -4127,7 +4127,14 @@ __initcall(slab_sysfs_init);
    /*
    * The /proc/slabinfo ABI
    */
    -#ifdef CONFIG_PROC_FS
    +#ifdef CONFIG_SLABINFO
    +
    +ssize_t slabinfo_write(struct file *file, const char __user * buffer,
    + size_t count, loff_t *ppos)
    +{
    + return -EINVAL;
    +}
    +

    static void print_slabinfo_header(struct seq_file *m)
    {
    @@ -4201,4 +4208,4 @@ const struct seq_operations slabinfo_op = {
    .show = s_show,
    };

    -#endif /* CONFIG_PROC_FS */
    +#endif /* CONFIG_SLABINFO */
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  5. Re: [PATCH] procfs: provide slub's /proc/slabinfo



    On Wed, 2 Jan 2008, Linus Torvalds wrote:
    >
    > Actually, I'd much rather just do this instead (on top of your patch)


    Side note - and I didn't do this - this also allows you to disable
    slabinfo for those people who think it is pointless.

    In particular, you could just make that

    > +config SLABINFO
    > + bool
    > + depends on PROC_FS
    > + depends on SLAB || SLUB
    > + default y


    Kconfig entry use something like

    bool "Enable /proc/slabinfo support"

    (perhaps adding an "if EMBEDDED" at the end), and now people can decide
    whether they actually want it or not (independently of whether they use
    SLUB or SLAB or whatever). But I left it hardcoded to on, just because I'm
    not sure it's worth it.

    Linus
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  6. Re: [PATCH] procfs: provide slub's /proc/slabinfo

    Hi Linus,

    On Jan 2, 2008 9:35 PM, Linus Torvalds wrote:
    > Actually, I'd much rather just do this instead (on top of your patch)
    >
    > It just creates a new CONFIG_SLABINFO that automatically has the right
    > dependencies (ie depends on PROC being on, and either SLAB or SLUB), and
    > then both SLAB and SLUB just have the exact same interfaces.
    >
    > Which means that SLOB could also trivially implement the same thing, with
    > no new #ifdef'fery or other crud.
    >
    > It's totally untested, but looks fairly obvious.


    Looks good to me. Thanks! :-)

    Reviewed-by: Pekka Enberg
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  7. Re: [PATCH] procfs: provide slub's /proc/slabinfo


    On Wed, 2008-01-02 at 11:35 -0800, Linus Torvalds wrote:
    >
    > On Wed, 2 Jan 2008, Pekka Enberg wrote:
    > >
    > > I already sent the remaining bits to Linus but this looks much
    > > cleaner. Thanks Hugh!
    > >
    > > Acked-by: Pekka Enberg

    >
    > Actually, I'd much rather just do this instead (on top of your patch)
    >
    > It just creates a new CONFIG_SLABINFO that automatically has the right
    > dependencies (ie depends on PROC being on, and either SLAB or SLUB), and
    > then both SLAB and SLUB just have the exact same interfaces.
    >
    > Which means that SLOB could also trivially implement the same thing, with
    > no new #ifdef'fery or other crud.


    Except SLOB's emulation of slabs is so thin, it doesn't have the
    relevant information. We have a very small struct kmem_cache, which I
    suppose could contain a counter. But we don't have anything like the
    kmalloc slabs, so you'd only be getting half the picture anyway.
    The output of slabtop would simply be misleading because there are no
    underlying "slabs" in the first place.

    --
    Mathematics is the supreme nostalgia of our time.

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  8. Re: [PATCH] procfs: provide slub's /proc/slabinfo


    * Matt Mackall wrote:

    > > Which means that SLOB could also trivially implement the same thing,
    > > with no new #ifdef'fery or other crud.

    >
    > Except SLOB's emulation of slabs is so thin, it doesn't have the
    > relevant information. We have a very small struct kmem_cache, which I
    > suppose could contain a counter. But we don't have anything like the
    > kmalloc slabs, so you'd only be getting half the picture anyway. The
    > output of slabtop would simply be misleading because there are no
    > underlying "slabs" in the first place.


    i think SLOB/embedded is sufficiently special that a "no /proc/slabinfo"
    restriction is perfectly supportable. (for instance it's only selectable
    if CONFIG_EMBEDDED=y) If a SLOB user has any memory allocation problems
    it's worth going to the bigger allocators anyway, to get all the
    debugging goodies.

    btw., do you think it would be worth/possible to have build mode for
    SLUB that is acceptably close to the memory efficiency of SLOB? (and
    hence work towards unifying all the 3 allocators into SLUB in essence)

    right now we are far away from it - SLUB has an order of magnitude
    larger .o than SLOB, even on UP. I'm wondering why that is so - SLUB's
    data structures _are_ quite compact and could in theory be used in a
    SLOB-alike way. Perhaps one problem is that much of SLUB's debugging
    code is always built in?

    Ingo
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  9. Re: [PATCH] procfs: provide slub's /proc/slabinfo


    On Thu, 2008-01-03 at 09:52 +0100, Ingo Molnar wrote:
    > * Matt Mackall wrote:
    >
    > > > Which means that SLOB could also trivially implement the same thing,
    > > > with no new #ifdef'fery or other crud.

    > >
    > > Except SLOB's emulation of slabs is so thin, it doesn't have the
    > > relevant information. We have a very small struct kmem_cache, which I
    > > suppose could contain a counter. But we don't have anything like the
    > > kmalloc slabs, so you'd only be getting half the picture anyway. The
    > > output of slabtop would simply be misleading because there are no
    > > underlying "slabs" in the first place.

    >
    > i think SLOB/embedded is sufficiently special that a "no /proc/slabinfo"
    > restriction is perfectly supportable. (for instance it's only selectable
    > if CONFIG_EMBEDDED=y) If a SLOB user has any memory allocation problems
    > it's worth going to the bigger allocators anyway, to get all the
    > debugging goodies.
    >
    > btw., do you think it would be worth/possible to have build mode for
    > SLUB that is acceptably close to the memory efficiency of SLOB? (and
    > hence work towards unifying all the 3 allocators into SLUB in essence)


    There are three downsides with the slab-like approach: internal
    fragmentation, under-utilized slabs, and pinning.

    The first is the situation where we ask for a kmalloc of 33 bytes and
    get 64. I think the average kmalloc wastes about 30% trying to fit into
    power-of-two buckets. We can tune our buckets a bit, but I think in
    general trying to back kmalloc with slabs is problematic. SLOB has a
    2-byte granularity up to the point where it just hands things off to the
    page allocator.

    If we tried to add more slabs to fill the gaps, we'd exacerbate the
    second problem: because only one type of object can go on a slab, a lot
    of slabs are half-full. SLUB's automerging of slabs helps some here, but
    is still restricted to objects of the same size.

    And finally, there's the whole pinning problem: we can have a cache like
    the dcache grow very large and then contract, but still have most of its
    slabs used by pinned dentries. Christoph has some rather hairy patches
    to address this, but SLOB doesn't have much of a problem here - those
    pages are still available to allocate other objects on.

    By comparison, SLOB's big downsides are that it's not O(1) and it has a
    single lock. But it's currently fast enough to keep up with SLUB on
    kernel compiles on my 2G box and Nick had an allocator benchmark where
    scalability didn't fall off until beyond 4 CPUs.

    > right now we are far away from it - SLUB has an order of magnitude
    > larger .o than SLOB, even on UP. I'm wondering why that is so - SLUB's
    > data structures _are_ quite compact and could in theory be used in a
    > SLOB-alike way. Perhaps one problem is that much of SLUB's debugging
    > code is always built in?


    I think we should probably just accept that it makes sense to have more
    than one allocator. A 64MB single CPU machine is very, very different
    than a 64TB 4096-CPU machine. On one of those, it probably makes some
    sense to burn some memory for maximum scalability.

    --
    Mathematics is the supreme nostalgia of our time.

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  10. Re: [PATCH] procfs: provide slub's /proc/slabinfo

    On Thu, 3 Jan 2008, Ingo Molnar wrote:

    > right now we are far away from it - SLUB has an order of magnitude
    > larger .o than SLOB, even on UP. I'm wondering why that is so - SLUB's
    > data structures _are_ quite compact and could in theory be used in a
    > SLOB-alike way. Perhaps one problem is that much of SLUB's debugging
    > code is always built in?


    In embedded mode you can switch the debugging code off.

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  11. Re: [PATCH] procfs: provide slub's /proc/slabinfo

    On Thu, 3 Jan 2008, Matt Mackall wrote:

    > There are three downsides with the slab-like approach: internal
    > fragmentation, under-utilized slabs, and pinning.
    >
    > The first is the situation where we ask for a kmalloc of 33 bytes and
    > get 64. I think the average kmalloc wastes about 30% trying to fit into
    > power-of-two buckets. We can tune our buckets a bit, but I think in
    > general trying to back kmalloc with slabs is problematic. SLOB has a
    > 2-byte granularity up to the point where it just hands things off to the
    > page allocator.


    The 2 byte overhead of SLOB becomes a liability when it comes to correctly
    aligning power of two sized object. SLOB has to add two bytes and then
    align the combined object (argh!). SLUB can align these without a 2 byte
    overhead. In some configurations this results in SLUB using even less
    memory than SLOB. See f.e. Pekka's test at
    http://marc.info/?l=linux-kernel&m=118405559214029&w=2

    > If we tried to add more slabs to fill the gaps, we'd exacerbate the
    > second problem: because only one type of object can go on a slab, a lot
    > of slabs are half-full. SLUB's automerging of slabs helps some here, but
    > is still restricted to objects of the same size.


    The advantage of SLOB is to be able to put objects of multiple sizes into
    the same slab page. That advantage goes away once we have more than a few
    objects per slab because SLUB can store object in a denser way than SLOB.

    > And finally, there's the whole pinning problem: we can have a cache like
    > the dcache grow very large and then contract, but still have most of its
    > slabs used by pinned dentries. Christoph has some rather hairy patches
    > to address this, but SLOB doesn't have much of a problem here - those
    > pages are still available to allocate other objects on.


    Well if you just have a few dentries then they are likely all pinned. A
    large number of dentries will typically result in reclaimable slabs.
    The slab defrag patchset not only deals with the dcache issue but provides
    similar solutions for inode and buffer_heads. Support for other slabs that
    defragment can be added by providing two hooks per slab.

    > By comparison, SLOB's big downsides are that it's not O(1) and it has a
    > single lock. But it's currently fast enough to keep up with SLUB on
    > kernel compiles on my 2G box and Nick had an allocator benchmark where
    > scalability didn't fall off until beyond 4 CPUs.


    Both SLOB and SLAB suffer from the single lock problem. SLOB does it for
    every item allocated. SLAB does it for every nth item allocated. Given
    a fast allocation from multiple processors both will generate a bouncing
    cacheline. SLUB can take pages from the page allocator pools and allocate
    all objects from it without taking a lock.

    > > right now we are far away from it - SLUB has an order of magnitude
    > > larger .o than SLOB, even on UP. I'm wondering why that is so - SLUB's
    > > data structures _are_ quite compact and could in theory be used in a
    > > SLOB-alike way. Perhaps one problem is that much of SLUB's debugging
    > > code is always built in?

    >
    > I think we should probably just accept that it makes sense to have more
    > than one allocator. A 64MB single CPU machine is very, very different
    > than a 64TB 4096-CPU machine. On one of those, it probably makes some
    > sense to burn some memory for maximum scalability.


    I still have trouble to see that SLOB still has much to offer. An embedded
    allocator that in many cases has more allocation overhead than the default
    one? Ok you still have advantages if allocations are rounded up to the
    next power of two for a kmalloc and because of the combining of different
    types of allocations in a single slab if there are an overall small number
    of allocations. If one would create a custom slab for the worst problems
    there then this may also go away.
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  12. Re: [PATCH] procfs: provide slub's /proc/slabinfo

    > I still have trouble to see that SLOB still has much to offer. An embedded
    > allocator that in many cases has more allocation overhead than the default
    > one? Ok you still have advantages if allocations are rounded up to the
    > next power of two for a kmalloc and because of the combining of different
    > types of allocations in a single slab if there are an overall small number
    > of allocations. If one would create a custom slab for the worst problems
    > there then this may also go away.


    I suspect it would be a good idea anyways to reevaluate the power of two
    slabs. Perhaps a better distribution can be found based on some profiling?
    I did profile kmalloc using a systemtap script some time ago but don't
    remember the results exactly, but iirc it looked like it could be improved.

    A long time ago i also had some code to let the network stack give hints
    about its MMUs to slab to create fitting slabs for packets. But that
    was never really pushed forward because it turned out it didn't help
    much for the most common 1.5K MTU -- always only two packets fit into
    a page.

    -Andi
    >

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  13. Re: [PATCH] procfs: provide slub's /proc/slabinfo


    On Thu, 2008-01-03 at 18:21 -0800, Christoph Lameter wrote:
    > On Thu, 3 Jan 2008, Matt Mackall wrote:
    >
    > > There are three downsides with the slab-like approach: internal
    > > fragmentation, under-utilized slabs, and pinning.
    > >
    > > The first is the situation where we ask for a kmalloc of 33 bytes and
    > > get 64. I think the average kmalloc wastes about 30% trying to fit into
    > > power-of-two buckets. We can tune our buckets a bit, but I think in
    > > general trying to back kmalloc with slabs is problematic. SLOB has a
    > > 2-byte granularity up to the point where it just hands things off to the
    > > page allocator.

    >
    > The 2 byte overhead of SLOB becomes a liability when it comes to correctly
    > aligning power of two sized object. SLOB has to add two bytes and then
    > align the combined object (argh!).


    It does no such thing. Only kmallocs have 2-byte headers and kmalloc is
    never aligned. RTFS, it's quite short.

    > SLUB can align these without a 2 byte
    > overhead. In some configurations this results in SLUB using even less
    > memory than SLOB. See f.e. Pekka's test at
    > http://marc.info/?l=linux-kernel&m=118405559214029&w=2


    Available memory after boot is not a particularly stable measurement and
    not valid if there's memory pressure. At any rate, I wasn't able to
    reproduce this.

    > > If we tried to add more slabs to fill the gaps, we'd exacerbate the
    > > second problem: because only one type of object can go on a slab, a lot
    > > of slabs are half-full. SLUB's automerging of slabs helps some here, but
    > > is still restricted to objects of the same size.

    >
    > The advantage of SLOB is to be able to put objects of multiple sizes into
    > the same slab page. That advantage goes away once we have more than a few
    > objects per slab because SLUB can store object in a denser way than SLOB.


    Ugh, Christoph. Can you please stop repeating this falsehood? I'm sick
    and tired of debunking it. There is no overhead for any objects with
    externally-known size. So unless SLUB actually has negative overhead,
    this just isn't true.

    > > And finally, there's the whole pinning problem: we can have a cache like
    > > the dcache grow very large and then contract, but still have most of its
    > > slabs used by pinned dentries. Christoph has some rather hairy patches
    > > to address this, but SLOB doesn't have much of a problem here - those
    > > pages are still available to allocate other objects on.

    >
    > Well if you just have a few dentries then they are likely all pinned. A
    > large number of dentries will typically result in reclaimable slabs.
    > The slab defrag patchset not only deals with the dcache issue but provides
    > similar solutions for inode and buffer_heads. Support for other slabs that
    > defragment can be added by providing two hooks per slab.


    What's your point? Slabs have a inherent pinning problem that's ugly to
    combat. SLOB doesn't.

    > > By comparison, SLOB's big downsides are that it's not O(1) and it has a
    > > single lock. But it's currently fast enough to keep up with SLUB on
    > > kernel compiles on my 2G box and Nick had an allocator benchmark where
    > > scalability didn't fall off until beyond 4 CPUs.

    >
    > Both SLOB and SLAB suffer from the single lock problem. SLOB does it for
    > every item allocated. SLAB does it for every nth item allocated. Given
    > a fast allocation from multiple processors both will generate a bouncing
    > cacheline. SLUB can take pages from the page allocator pools and allocate
    > all objects from it without taking a lock.


    That's very nice, but SLOB already runs lockless on most of its target
    machines by virtue of them being UP. Lock contention just isn't
    interesting to SLOB.

    > > > right now we are far away from it - SLUB has an order of

    > magnitude
    > > > larger .o than SLOB, even on UP. I'm wondering why that is so -

    > SLUB's
    > > > data structures _are_ quite compact and could in theory be used in

    > a
    > > > SLOB-alike way. Perhaps one problem is that much of SLUB's

    > debugging
    > > > code is always built in?

    > >
    > > I think we should probably just accept that it makes sense to have

    > more
    > > than one allocator. A 64MB single CPU machine is very, very

    > different
    > > than a 64TB 4096-CPU machine. On one of those, it probably makes

    > some
    > > sense to burn some memory for maximum scalability.

    >
    > I still have trouble to see that SLOB still has much to offer. An
    > embedded
    > allocator that in many cases has more allocation overhead than the
    > default
    > one?


    For the benefit of anyone who didn't read this the last few times I
    rehashed this with you:

    SLOB:
    - internal overhead for kmalloc is 2 bytes (or 3 for odd-sized objects)
    - internal overhead for kmem_cache_alloc is 0 bytes (or 1 for odd-sized
    objects)
    - any unused space down to 2 bytes on any SLOB page can be allocated by
    any object that will fit

    SLAB/SLUB
    - internal overhead for kmalloc averages about 30%
    - internal overhead for kmem_cache_alloc is (slab-size % object-size) /
    objects-per-slab, which can be quite large for things like SKBs and
    task_structs and is made worse by alignment
    - unused space on slabs can't be allocated by objects of other
    sizes/types

    The only time SLAB/SLUB can win in efficiency (assuming they're using
    the same page size) is when all your kmallocs just happen to be powers
    of two. Which, assuming any likely distribution of string or other
    object sizes, isn't often.

    --
    Mathematics is the supreme nostalgia of our time.

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  14. Re: [PATCH] procfs: provide slub's /proc/slabinfo


    On Fri, 2008-01-04 at 03:45 +0100, Andi Kleen wrote:
    > > I still have trouble to see that SLOB still has much to offer. An embedded
    > > allocator that in many cases has more allocation overhead than the default
    > > one? Ok you still have advantages if allocations are rounded up to the
    > > next power of two for a kmalloc and because of the combining of different
    > > types of allocations in a single slab if there are an overall small number
    > > of allocations. If one would create a custom slab for the worst problems
    > > there then this may also go away.

    >
    > I suspect it would be a good idea anyways to reevaluate the power of two
    > slabs. Perhaps a better distribution can be found based on some profiling?
    > I did profile kmalloc using a systemtap script some time ago but don't
    > remember the results exactly, but iirc it looked like it could be improved.


    We can roughly group kmalloced objects into two classes:

    a) intrinsically variable-sized (strings, etc.)
    b) fixed-sized objects that nonetheless don't have their own caches

    For (a), we can expect the size distribution to be approximately a
    scale-invariant power distribution. So buckets of the form n**x make a
    fair amount of sense. We might consider n less than 2 though.

    For objects of type (b) that occur in significant numbers, well, we
    might just want to add more caches. SLUB's merging of same-sized caches
    will reduce the pain here.

    > A long time ago i also had some code to let the network stack give hints
    > about its MMUs to slab to create fitting slabs for packets. But that
    > was never really pushed forward because it turned out it didn't help
    > much for the most common 1.5K MTU -- always only two packets fit into
    > a page.


    Yes, that and task_struct kinda make you want to cry. Large-order
    SLAB/SLUB/SLOB would go a long way to fix that, but has its own problems
    of course.

    One could imagine restructuring things so that the buddy allocator only
    extended down to 64k or so and below that, gfp and friends called
    through SLAB/SLUB/SLOB.

    --
    Mathematics is the supreme nostalgia of our time.

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  15. Re: [PATCH] procfs: provide slub's /proc/slabinfo


    On Fri, 2008-01-04 at 03:45 +0100, Andi Kleen wrote:
    > > I still have trouble to see that SLOB still has much to offer. An embedded
    > > allocator that in many cases has more allocation overhead than the default
    > > one? Ok you still have advantages if allocations are rounded up to the
    > > next power of two for a kmalloc and because of the combining of different
    > > types of allocations in a single slab if there are an overall small number
    > > of allocations. If one would create a custom slab for the worst problems
    > > there then this may also go away.

    >
    > I suspect it would be a good idea anyways to reevaluate the power of two
    > slabs. Perhaps a better distribution can be found based on some profiling?
    > I did profile kmalloc using a systemtap script some time ago but don't
    > remember the results exactly, but iirc it looked like it could be improved.


    I remember wli trying to work out a series that had minimal
    fragmentation. IIRC he was mixing a fibonaci series with the power of
    two series.

    Bill, do you remember getting anywhere?

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQBHffmrXA2jU0ANEf4RArGUAJwJDIuK+J1IdaUlfR5OX7 hqJ4ugtQCeO2/I
    s/6cUfN/iTP3y69Nr2HeNhM=
    =frz6
    -----END PGP SIGNATURE-----


  16. Re: [PATCH] procfs: provide slub's /proc/slabinfo

    On Thu, 3 Jan 2008, Matt Mackall wrote:

    > > The advantage of SLOB is to be able to put objects of multiple sizes into
    > > the same slab page. That advantage goes away once we have more than a few
    > > objects per slab because SLUB can store object in a denser way than SLOB.

    >
    > Ugh, Christoph. Can you please stop repeating this falsehood? I'm sick
    > and tired of debunking it. There is no overhead for any objects with
    > externally-known size. So unless SLUB actually has negative overhead,
    > this just isn't true.


    Hmmm.. Seems that I still do not understand how it is possible then to mix
    objects of different sizes in the same slab page. Somehow the allocator
    needs to know the size. So it is not possible in SLOB to use
    kmem_cache_alloc on an object and then free it using kfree?

    > > Well if you just have a few dentries then they are likely all pinned. A
    > > large number of dentries will typically result in reclaimable slabs.
    > > The slab defrag patchset not only deals with the dcache issue but provides
    > > similar solutions for inode and buffer_heads. Support for other slabs that
    > > defragment can be added by providing two hooks per slab.

    >
    > What's your point? Slabs have a inherent pinning problem that's ugly to
    > combat. SLOB doesn't.


    I thought we were talking about pinning problems of dentries. How are
    slabs pinned and why does it matter? If slabs are pineed by a dentry that
    is pinned then the slab page will be filled up with other dentries that
    are not pinned. The slab defrag approach causes a coalescing of objects
    around slabs that have pinned objects.

    > SLOB:
    > - internal overhead for kmalloc is 2 bytes (or 3 for odd-sized objects)


    Well that increase if you need to align the object. For kmalloc this
    usually means cache line align a power of two object right? So we have a
    cacheline size of overhead?

    > - internal overhead for kmem_cache_alloc is 0 bytes (or 1 for odd-sized
    > objects)


    You are not aligning to a double word boundary? This will create issues on
    certain platforms.

    > SLAB/SLUB
    > - internal overhead for kmalloc averages about 30%


    I think that is valid for a random object size distribution?

    > - internal overhead for kmem_cache_alloc is (slab-size % object-size) /
    > objects-per-slab, which can be quite large for things like SKBs and
    > task_structs and is made worse by alignment


    Good, so SLOB can fit in small objects in those holes.

    The calculation for SLAB is different since it also typically places it
    management structure in the slab. The management structure needs at least
    2 bytes per object.

    So the per object overhead in SLAB is

    ((slab-size - management-structure-overhead) % object-size) / objects-per-slab

    > The only time SLAB/SLUB can win in efficiency (assuming they're using
    > the same page size) is when all your kmallocs just happen to be powers
    > of two. Which, assuming any likely distribution of string or other
    > object sizes, isn't often.


    In case of SLAB that is true. In case of SLUB we could convert the
    kmallocs to kmem_cache_alloc. The newly created slab would in all
    likelyhood be an alias of an already existing structure and thus be
    essentially free. In that fashion SLUB can (in a limited way) put objects
    for different slab caches into the same slab page too.

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  17. Re: [PATCH] procfs: provide slub's /proc/slabinfo

    On Fri, 4 Jan 2008, Peter Zijlstra wrote:

    > I remember wli trying to work out a series that had minimal
    > fragmentation. IIRC he was mixing a fibonaci series with the power of
    > two series.
    >
    > Bill, do you remember getting anywhere?


    I tried various approaches to reduce the overhead that power of two slab
    sizes cause in SLUB during the initial comparison of memory use with SLOB.
    This involved creating slabs in 32 byte increments and trying to add a few
    additional extra slabs in between the power of two sizes. None of that led
    to convincing results.

    I found that the SLAB scheme with power of two caches and two extra ones
    (96 and 192 bytes) was optimal.
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  18. Re: [PATCH] procfs: provide slub's /proc/slabinfo


    On Fri, 2008-01-04 at 12:34 -0800, Christoph Lameter wrote:
    > On Thu, 3 Jan 2008, Matt Mackall wrote:
    >
    > > > The advantage of SLOB is to be able to put objects of multiple sizes into
    > > > the same slab page. That advantage goes away once we have more than a few
    > > > objects per slab because SLUB can store object in a denser way than SLOB.

    > >
    > > Ugh, Christoph. Can you please stop repeating this falsehood? I'm sick
    > > and tired of debunking it. There is no overhead for any objects with
    > > externally-known size. So unless SLUB actually has negative overhead,
    > > this just isn't true.

    >
    > Hmmm.. Seems that I still do not understand how it is possible then to mix
    > objects of different sizes in the same slab page. Somehow the allocator
    > needs to know the size. So it is not possible in SLOB to use
    > kmem_cache_alloc on an object and then free it using kfree?


    Indeed. Mismatching allocator and deallocator is bug, even if it happens
    to work for SLAB/SLUB.

    > > > Well if you just have a few dentries then they are likely all pinned. A
    > > > large number of dentries will typically result in reclaimable slabs.
    > > > The slab defrag patchset not only deals with the dcache issue but provides
    > > > similar solutions for inode and buffer_heads. Support for other slabs that
    > > > defragment can be added by providing two hooks per slab.

    > >
    > > What's your point? Slabs have a inherent pinning problem that's ugly to
    > > combat. SLOB doesn't.

    >
    > I thought we were talking about pinning problems of dentries. How are
    > slabs pinned and why does it matter?


    If a slab contains a dentry that is pinned, it can only be used for
    other dentries and cannot be recycled for other allocations. If updatedb
    comes along and fills memory with dentry slabs, many of which get
    permanently pinned, then you have wasted memory.

    > If slabs are pineed by a dentry that
    > is pinned then the slab page will be filled up with other dentries that
    > are not pinned. The slab defrag approach causes a coalescing of objects
    > around slabs that have pinned objects.


    Yes. You've got (most of) a fix. It's overly-complicated and SLOB
    doesn't need it. How many ways do I need to say this?


    > > SLOB:
    > > - internal overhead for kmalloc is 2 bytes (or 3 for odd-sized objects)

    >
    > Well that increase if you need to align the object. For kmalloc this
    > usually means cache line align a power of two object right? So we have a
    > cacheline size of overhead?


    a) alignment doesn't increase memory use because the memory before the
    object is still allocatable
    b) kmallocs aren't aligned!

    > > - internal overhead for kmem_cache_alloc is 0 bytes (or 1 for odd-sized
    > > objects)

    >
    > You are not aligning to a double word boundary? This will create issues on
    > certain platforms.


    The alignment minimum varies per arch. SLOB can go down to 2 bytes.

    > > SLAB/SLUB
    > > - internal overhead for kmalloc averages about 30%

    >
    > I think that is valid for a random object size distribution?


    It's a measurement from memory. But it roughly agrees with what you'd
    expect from a random distribution.

    > > The only time SLAB/SLUB can win in efficiency (assuming they're using
    > > the same page size) is when all your kmallocs just happen to be powers
    > > of two. Which, assuming any likely distribution of string or other
    > > object sizes, isn't often.

    >
    > In case of SLAB that is true. In case of SLUB we could convert the
    > kmallocs to kmem_cache_alloc. The newly created slab would in all
    > likelyhood be an alias of an already existing structure and thus be
    > essentially free. In that fashion SLUB can (in a limited way) put objects
    > for different slab caches into the same slab page too.


    Uh, no. You'd need a new slab for every multiple of 2 bytes. And then
    you'd just be making the underused and pinned slab problems worse.

    --
    Mathematics is the supreme nostalgia of our time.

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  19. Re: [PATCH] procfs: provide slub's /proc/slabinfo

    On Fri, 4 Jan 2008, Matt Mackall wrote:

    > > needs to know the size. So it is not possible in SLOB to use
    > > kmem_cache_alloc on an object and then free it using kfree?

    >
    > Indeed. Mismatching allocator and deallocator is bug, even if it happens
    > to work for SLAB/SLUB.


    Was the kernel audited for this case? I saw some rather scary uses of slab
    objects for I/O purposes come up during SLUB development.

    > Yes. You've got (most of) a fix. It's overly-complicated and SLOB
    > doesn't need it. How many ways do I need to say this?


    So SLOB is then not able to compact memory after an updatedb run? The
    memory must stay dedicated to slab uses. SLOB memory can be filled
    up with objects of other slab types but it cannot be reused for page
    cache and anonymous pages etc. Slab defrag is freeing memory back to the
    page allocator with SLUB. That is *not* provided by SLOB. Could be made to
    work though.

    > > Well that increase if you need to align the object. For kmalloc this
    > > usually means cache line align a power of two object right? So we have a
    > > cacheline size of overhead?

    >
    > a) alignment doesn't increase memory use because the memory before the
    > object is still allocatable


    Ok so we end up with lots of small holes on a list that has to be scanned
    to find free memory?

    > b) kmallocs aren't aligned!


    From mm/slob.c:

    #ifndef ARCH_KMALLOC_MINALIGN
    #define ARCH_KMALLOC_MINALIGN __alignof__(unsigned long)
    #endif

    void *__kmalloc_node(size_t size, gfp_t gfp, int node)
    {
    unsigned int *m;
    int align = max(ARCH_KMALLOC_MINALIGN, ARCH_SLAB_MINALIGN);

    (IMHO it would be safer to set the minimum default to __alignof__(unsigned
    long long) like SLAB/SLUB).



    Ok. So lets try a worst case scenario. If we do a 128 byte kmalloc then we
    can allocate the following number of object from one 4k slab

    SLUB 32 (all memory of the 4k page is used for 128 byte objects)
    SLAB 29/30 (management structure occupies first two/three objects)
    SLOB 30(?) (Alignment results in object being 136 byte of effective size,
    we have 16 bytes leftover that could be used for a
    very small allocation. Right?)
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  20. Re: [PATCH] procfs: provide slub's /proc/slabinfo


    On Fri, 2008-01-04 at 13:36 -0800, Christoph Lameter wrote:
    > Ok. So lets try a worst case scenario. If we do a 128 byte kmalloc
    > then we
    > can allocate the following number of object from one 4k slab
    >
    > SLUB 32 (all memory of the 4k page is used for 128 byte objects)
    > SLAB 29/30 (management structure occupies first two/three objects)
    > SLOB 30(?) (Alignment results in object being 136 byte of effective size,
    > we have 16 bytes leftover that could be used for a
    > very small allocation. Right?)


    Don't know how you got to 136, the minimum alignment is 4 on x86. But I
    already said in my last email that SLUB would win for the special case
    of power of two allocations. But as long as we're looking at worst
    cases, let's consider an alloc of 257 bytes..

    SLUB 8 (1016 bytes wasted)
    SLOB 15 (105 bytes wasted, with 136 bytes still usable)

    Can we be done with this now, please?

    --
    Mathematics is the supreme nostalgia of our time.

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

+ Reply to Thread
Page 1 of 4 1 2 3 ... LastLast