Problems constructing ELF program headers - Linux

This is a discussion on Problems constructing ELF program headers - Linux ; I'm trying to write a simple ELF executable construction utility for a compiler toolchain I'm working on ( http://tack.sf.net ). The ELF documentation documents the program header format quite straightforwardly. Unfortunately, I'm finding that the Linux kernel is considerably pickier ...

+ Reply to Thread
Results 1 to 12 of 12

Thread: Problems constructing ELF program headers

  1. Problems constructing ELF program headers

    I'm trying to write a simple ELF executable construction utility for a
    compiler toolchain I'm working on ( http://tack.sf.net ). The ELF
    documentation documents the program header format quite straightforwardly.
    Unfortunately, I'm finding that the Linux kernel is considerably pickier about
    what it thinks is a valid executable.

    Here's a simple example on i386:

    start address 0x08048054
    Program Header:
    LOAD off 0x00000000 vaddr 0x08048000 paddr 0x00000000 align 2**0
    filesz 0x000002b4 memsz 0x00000340 flags rwx

    This works.

    If, however, I try to change the file offset to 0x54, decreasing the memory
    size and file size accordingly (so that I don't end up loading the ELF header
    itself into memory), it won't load --- I just get 'Killed'. It would appear
    that the Linux ELF loader insists that the first loadable section starts at
    offset 0. Likewise, if the file size is not a multiple of 4, it also won't load.

    In particular, I'm finding that if I try to have more than one section, so
    that I can put my text and data segments in different loadable sections with
    different access flags, frequently it'll just fail, and I can't figure out why.

    I don't mind the fact that the Linux kernel only supports a subset of the
    standard, but I *do* mind that this doesn't appear to be documented anywhere.
    I've tried looking at the kernel source, but the ELF loader is not
    particularly comprehensible... am I just looking in the wrong places? Can
    anyone point me at any documents describing exactly what the Linux kernel
    considers to be a valid ELF executable? Alternatively, if anyone knows how I
    can persuade the kernel to tell me *why* it's rejecting a particular file, I'd
    be much obliged...

    --
    ┌── dg*cowlark.com ─── http://www.cowlark.com ──────────────── ──

    │ "The first 90% of the code takes the first 90% of the time. The other 10%
    │ takes the other 90% of the time." --- Anonymous

  2. Re: Problems constructing ELF program headers

    Bilgehan.Balban@gmail.com wrote:
    [...]
    > Does your linker functionality overlap with what LD does? Probably
    > you've done it already, but if I were you I would compare the linker
    > output against what LD produces for the same sources. I would compile
    > an extremely simple source file, e.g.


    I've tried that; unfortunately, ld binaries are really rather complex,
    containing all kinds of weird sections (STACK program sections, for example,
    which are not mentioned in the ELF spec at all), as well as full segment
    tables, which ought to be irrelevant for executables.

    I've been looking at the Whirlwind Guide To Minimal ELF Executables, which has
    got a complete example. That layout has a single rx section, which is what I'm
    using now and it works, but it would be really nice to be able to make my text
    segment read-only.

    ....oh, yes, and most of objdump's functionality only works with segments, not
    phdr sections. I can't disassemble or dump my binaries, for example. At least
    it'll dump the program header.

    --
    ┌── dg*cowlark.com ─── http://www.cowlark.com ──────────────── ──

    │ "The first 90% of the code takes the first 90% of the time. The other 10%
    │ takes the other 90% of the time." --- Anonymous

  3. Re: Problems constructing ELF program headers

    On Mar 1, 3:48 pm, David Given wrote:
    > I've tried that; unfortunately, ld binaries are really rather complex,
    > containing all kinds of weird sections (STACK program sections, for example,
    > which are not mentioned in the ELF spec at all), as well as full segment
    > tables, which ought to be irrelevant for executables.


    I forgot to say, the default linker script ld uses is not good for
    this. If you pass your own script to LD, you can obtain a minimal
    executable. E.g. try:
    ENTRY(_start)
    SECTIONS
    {
    .text
    {
    *(.text)
    }
    }

    also add .rodata .data and .bss the same way. You will see you only
    get those sections you specify.

    > ...oh, yes, and most of objdump's functionality only works with segments, not
    > phdr sections. I can't disassemble or dump my binaries, for example. At least
    > it'll dump the program header.
    >


    The other thing I could advice is try this:
    http://www.disy.cse.unsw.edu.au/lxr/...e/tools/pyelf/ if
    you know python well, it is a tool that does most of what you can do
    with readelf, but it is written in python, so very easy to tweak, and
    peek what phdrs, sections are being read by the program, and how. See
    elf.py for example. It works fine on usual LD executables, so gives
    you a chance to compare what it reads. If you dont know python,
    learning is very very easy. I used this tool myself to extract stuff
    from elf files.


    Bahadir


  4. Re: Problems constructing ELF program headers

    In article ,
    David Given wrote:
    >
    >start address 0x08048054
    >Program Header:
    > LOAD off 0x00000000 vaddr 0x08048000 paddr 0x00000000 align 2**0
    > filesz 0x000002b4 memsz 0x00000340 flags rwx
    >
    >This works.
    >
    >If, however, I try to change the file offset to 0x54, decreasing the memory
    >size and file size accordingly (so that I don't end up loading the ELF header
    >itself into memory),


    Trying to save 84 bytes of memory per process? Besides being a symptom of
    OCD, this is not possible, since memory comes in chunks of 4096 bytes called
    pages.

    The ELF program header is described in Chapter 5 of the System V ABI[1]. It
    says:

    As "Program Loading" describes in this chapter of the processor supplement,
    loadable process segments must have congruent values for p_vaddr and
    p_offset, modulo the page size.

    This restriction on p_vaddr and p_offset is awkwardly placed in the
    description of p_align, but if you look in the processor supplement (for
    example the Intel386 processor supplement[2]) it's more specific:

    Virtual addresses and and file offsets for the Intel386 architecture
    segments are congruent modulo 4 KB (0x1000) or larger powers of 2.

    If your offset ends in 054, your vaddr must also end in 054. But that won't
    trick the kernel into not mapping the first 0x54 bytes, it'll map them
    anyway. It has to. Loading an ELF image is basically equivalent to a series
    of mmap calls with parameters taken from the program header, and mmap only
    works on pages. There is no mechanism for mapping a portion of a file that
    isn't aligned on a page boundary.

    If your file size is 0x1054 bytes and you want to map only the last 0x1000
    bytes, tough luck. Your only choices are to put padding in the file so that
    the part you want to map starts at offset 0x1000, or map the whole file (and
    get a 0x2000-byte process image, including 0xfac bytes of trailing zeros).

    [1] http://refspecs.freestandards.org/elf/gabi41.pdf
    [2] http://refspecs.freestandards.org/elf/abi386-4.pdf

    >
    >I don't mind the fact that the Linux kernel only supports a subset of the
    >standard, but I *do* mind that this doesn't appear to be documented anywhere.


    There's a real standard document that says what you did wrong... is that good
    enough?

    --
    Alan Curry
    pacman@world.std.com

  5. Re: Problems constructing ELF program headers

    On 1 Mar, 21:50, pac...@TheWorld.com (Alan Curry) wrote:
    > The ELF program header is described in Chapter 5 of the System V ABI[1]. It
    > says:
    >
    > As "Program Loading" describes in this chapter of the processor supplement,
    > loadable process segments must have congruent values for p_vaddr and
    > p_offset, modulo the page size.
    >
    > This restriction on p_vaddr and p_offset is awkwardly placed in the
    > description of p_align, but if you look in the processor supplement (for
    > example the Intel386 processor supplement[2]) it's more specific:
    >
    > Virtual addresses and and file offsets for the Intel386 architecture
    > segments are congruent modulo 4 KB (0x1000) or larger powers of 2.
    >
    > If your offset ends in 054, your vaddr must also end in 054.
    > --
    > Alan Curry
    > pac...@world.std.com


    I also got caught once by this. From the elf spec: p_align: ... This
    member gives the value to which the segments are aligned in memory and
    in the file. ... p_align should be a positive, integral power of 2,
    and p_addr should equal p_offset, modulo p_align.

    On the ARM architecture, using 4KB pages, this value in fact was 8KB,
    whereas I was expecting a 4KB page size would suffice, and the section
    I wanted to load was only a handful of bytes.

    Bahadir


  6. Re: Problems constructing ELF program headers

    On Mar 1, 10:29 pm, Bilgehan.Bal...@gmail.com wrote:

    > On the ARM architecture, using 4KB pages, this value in fact was 8KB,


    Just checked for correctness its 0x8000, so meant 32KB.

    Bahadir



  7. Re: Problems constructing ELF program headers

    Alan Curry wrote:
    [...]
    > If your offset ends in 054, your vaddr must also end in 054. But that won't
    > trick the kernel into not mapping the first 0x54 bytes, it'll map them
    > anyway. It has to. Loading an ELF image is basically equivalent to a series
    > of mmap calls with parameters taken from the program header, and mmap only
    > works on pages. There is no mechanism for mapping a portion of a file that
    > isn't aligned on a page boundary.


    Except that this is clearly not the case. From my Ubuntu Edgy i386 system,
    objdump -x /bin/cat gives:

    LOAD off 0x00000000 vaddr 0x08048000 paddr 0x08048000 align 2**12
    filesz 0x00003b94 memsz 0x00003b94 flags r-x
    LOAD off 0x00003b94 vaddr 0x0804cb94 paddr 0x0804cb94 align 2**12
    filesz 0x000001c8 memsz 0x00000314 flags rw-

    (heavily trimmed)

    Note the total absence of alignment.

    [...]
    > There's a real standard document that says what you did wrong... is that good
    > enough?


    No, it's not, because my file *does* comply with the standard, as far as I can
    tell. One of my test programs looked like this (from memory, so may contain
    typos):

    LOAD off 0x00000000 vaddr 0x08048000 paddr 0x08048000 align 2**12
    filesz 0x00001000 memsz 0x00001000 flags r-x
    LOAD off 0x00001000 vaddr 0x08049000 paddr 0x08049000 align 2**12
    filesz 0x00001000 memsz 0x00001000 flags rw-

    I really can't see anything wrong with that, so why does it fail?

    In fact, I don't care that Linux doesn't seem to support the letter of the ELF
    standard. What I do care is that it doesn't seem to be documented what it
    *does* support. Why does the first LOAD segment's off have to be 0? It doesn't
    say anything about that in the standard... nor the fact that filesz must be a
    multiple of 4... nor whatever it is I'm doing wrong in the above example. I
    can probably figure all this out by trial and error, but doing so is a total
    waste of my time. Where's the *documentation*?

    --
    ┌── dg*cowlark.com ─── http://www.cowlark.com ──────────────── ──

    │ "The first 90% of the code takes the first 90% of the time. The other 10%
    │ takes the other 90% of the time." --- Anonymous

  8. Re: Problems constructing ELF program headers

    David Given wrote:
    > Alan Curry wrote:
    > [...]
    >
    >>If your offset ends in 054, your vaddr must also end in 054. But that won't
    >>trick the kernel into not mapping the first 0x54 bytes, it'll map them
    >>anyway. It has to. Loading an ELF image is basically equivalent to a series
    >>of mmap calls with parameters taken from the program header, and mmap only
    >>works on pages. There is no mechanism for mapping a portion of a file that
    >>isn't aligned on a page boundary.

    >
    >
    > Except that this is clearly not the case. From my Ubuntu Edgy i386 system,
    > objdump -x /bin/cat gives:
    >
    > LOAD off 0x00000000 vaddr 0x08048000 paddr 0x08048000 align 2**12
    > filesz 0x00003b94 memsz 0x00003b94 flags r-x
    > LOAD off 0x00003b94 vaddr 0x0804cb94 paddr 0x0804cb94 align 2**12
    > filesz 0x000001c8 memsz 0x00000314 flags rw-
    >
    > (heavily trimmed)
    >
    > Note the total absence of alignment.


    Ahem: .p_vaddr === .p_offset (modulo (1<<.p_align)) .
    In this specific case, the .p_align is 12, implying 4KB alignment,
    which is the 3 least-significant hexadecimal digits. In the first PT_LOAD
    ..p_vaddr ends in 0x...000 and
    ..p_offset ends in 0x...000,
    which satisfies the alignment restriction. In the second PT_LOAD,
    ..p_vaddr ends in 0x...b94 and
    ..p_offset ends in 0x...b94,
    which satisfies the alignment restriction.

    >
    > [...]
    >
    >>There's a real standard document that says what you did wrong... is that good
    >>enough?

    >
    >
    > No, it's not, because my file *does* comply with the standard, as far as I can
    > tell. One of my test programs looked like this (from memory, so may contain
    > typos):
    >
    > LOAD off 0x00000000 vaddr 0x08048000 paddr 0x08048000 align 2**12
    > filesz 0x00001000 memsz 0x00001000 flags r-x
    > LOAD off 0x00001000 vaddr 0x08049000 paddr 0x08049000 align 2**12
    > filesz 0x00001000 memsz 0x00001000 flags rw-
    >
    > I really can't see anything wrong with that, so why does it fail?


    Consult the code at the end of this reply. It generates a program
    which matches the description above, and the generated program runs
    without failing on linux-2.6.17 for x86.

    >
    > In fact, I don't care that Linux doesn't seem to support the letter of the ELF
    > standard. What I do care is that it doesn't seem to be documented what it
    > *does* support. Why does the first LOAD segment's off have to be 0?


    The .p_offset of the first PT_LOAD does not have to be 0.

    It doesn't
    > say anything about that in the standard... nor the fact that filesz must be a
    > multiple of 4...


    The .p_filesz is not required to be a multiple of 4.

    nor whatever it is I'm doing wrong in the above example. I
    > can probably figure all this out by trial and error, but doing so is a total
    > waste of my time. Where's the *documentation*?


    The documentation is the code itself in [/usr/src/] linux/fs/binfmt_elf.c ,
    function load_elf_binary(). The code is much more lucid than any doc.
    Yes, the actual requirements [may] _change_ from kernel to kernel,
    particularly because they are security sensitive. A bug in interpreting
    the PT_LOAD, or PT_LOAD which try to change mappings of kernel-reserved
    address space, could result in privilege escalation, etc.

    -----
    #include
    #include
    #include

    unsigned const PAGE_SHIFT = 12;
    unsigned const PAGE_SIZE = (1<<12);
    unsigned char code[] = { /* x86 */
    0xbb, 0,0,0,0, /* status value */
    0xb8, 1,0,0,0, /* __NR_exit */
    0xcd, 0x80 /* syscall */
    };

    main()
    {
    Elf32_Ehdr ehdr;
    int const N_PHDR = 2;

    memset(&ehdr, 0, sizeof(ehdr));
    memcpy(ehdr.e_ident, ELFMAG, SELFMAG);
    ehdr.e_ident[EI_CLASS] = ELFCLASS32;
    ehdr.e_ident[EI_DATA] = ELFDATA2LSB;
    ehdr.e_ident[EI_VERSION] = EV_CURRENT;
    ehdr.e_ident[EI_OSABI] = ELFOSABI_LINUX;
    ehdr.e_ident[EI_ABIVERSION] = EV_CURRENT;
    ehdr.e_type = ET_EXEC;
    ehdr.e_machine = EM_386;
    ehdr.e_entry = 0x08048000 + 2*sizeof(Elf32_Phdr) + sizeof(ehdr);
    ehdr.e_phoff = sizeof(ehdr);
    ehdr.e_ehsize = sizeof(ehdr);
    ehdr.e_phentsize = sizeof(Elf32_Phdr);
    ehdr.e_phnum = N_PHDR;
    write(1, &ehdr, sizeof(ehdr));

    int j;
    for (j=0; j < 2; ++j) {
    Elf32_Phdr phdr;
    memset(&phdr, 0, sizeof(phdr));
    phdr.p_type = PT_LOAD;
    phdr.p_offset = PAGE_SIZE * j;
    phdr.p_vaddr = j * PAGE_SIZE + 0x08048000;
    phdr.p_paddr = phdr.p_vaddr;
    phdr.p_filesz = PAGE_SIZE;
    phdr.p_memsz = PAGE_SIZE;
    phdr.p_flags = ((0==j) ? (PF_X | PF_R) : (PF_W | PF_R));
    phdr.p_align = PAGE_SHIFT;
    write(1, &phdr, sizeof(phdr));
    }

    for (j=0; j < 2; ++j) {
    write(1, code, sizeof(code));
    lseek(1, (1+ j)< }
    return 0;
    }
    -----

    --

  9. Re: Problems constructing ELF program headers

    John Reiser wrote:
    [...]
    > Ahem: .p_vaddr === .p_offset (modulo (1<<.p_align)) .
    > In this specific case, the .p_align is 12, implying 4KB alignment,
    > which is the 3 least-significant hexadecimal digits. In the first PT_LOAD
    > .p_vaddr ends in 0x...000 and
    > .p_offset ends in 0x...000,
    > which satisfies the alignment restriction. In the second PT_LOAD,
    > .p_vaddr ends in 0x...b94 and
    > .p_offset ends in 0x...b94,
    > which satisfies the alignment restriction.


    Arrgh! *slaps forehead*

    For some reason I'd interpreted that as being a requirement on (vaddr + memsz)
    being aligned.

    [...]
    > The .p_offset of the first PT_LOAD does not have to be 0.


    Yeah, that makes sense --- but unfortunately it means that my particular
    requirement (to have code that's in the file *after* the ELF header show up at
    0x8048000) is now not achievable. Unless I'm willing to eat 4kB of dummy space
    at the beginning of the file. Hmm. Will think.

    Ta (knew there was something I was getting wrong)...

    --
    ┌── dg*cowlark.com ─── http://www.cowlark.com ──────────────── ──

    │ "The first 90% of the code takes the first 90% of the time. The other 10%
    │ takes the other 90% of the time." --- Anonymous

  10. Re: Problems constructing ELF program headers

    In article ,
    David Given wrote:
    >Yeah, that makes sense --- but unfortunately it means that my particular
    >requirement (to have code that's in the file *after* the ELF header show up at
    >0x8048000) is now not achievable. Unless I'm willing to eat 4kB of dummy space
    >at the beginning of the file. Hmm. Will think.
    >


    If you use ext3 with 4K blocks like most of us do, your file is already
    padded at the end to a multiple of 4K anyway. All you'll be doing is moving
    that padding to a place where it helps you. You won't consume any more disk
    space. (Check st_blocks, not st_size -- use ls -s or du, not ls -l)

    --
    Alan Curry
    pacman@world.std.com

  11. Re: Problems constructing ELF program headers

    I have some problems about ELF, I have done a simple test on editing a ELF executable file(statically linked), as ELF specification said, the PT_LOAD segment can be at any place in the file if it meet the requirement that p_offset = p_vaddr % p_align, what I did is just modify every program header's p_offset to p_offset+0x1000, and then seek to the file offset 0x1000, writing all the original segment conent into the file. I have used this way to do the same test on a simple helloworld program on a MIPS and a X86 machine respectively, the newly generated file can run successfully on MIPS machine, but failed on X86 machine (segmentation falt), do you know is there any different behavior for the elf loading process between MIPS and X86 machine?
    It is very kind of you to provide some advice, thank you very much!
    Wait for your reply.

  12. Re: Problems constructing ELF program headers

    I have some problems about ELF, I have done a simple test on editing a ELF executable file(statically linked), as ELF specification said, the PT_LOAD segment can be at any place in the file if it meet the requirement that p_offset = p_vaddr % p_align, what I did is just modify every program header's p_offset to p_offset+0x1000, and then seek to the file offset 0x1000, writing all the original segment conent into the file. I have used this way to do the same test on a simple helloworld program on a MIPS and a X86 machine respectively, the newly generated file can run successfully on MIPS machine, but failed on X86 machine (segmentation falt), do you know is there any different behavior for the elf loading process between MIPS and X86 machine?
    It is very kind of you to provide some advice, thank you very much!
    Wait for your reply.

+ Reply to Thread