memory mapped file question - Unix

This is a discussion on memory mapped file question - Unix ; Hi, We have some API's that accept files as one of the parameters, and these APIs either read or write into those files (FILE*s). Now we have a request to support a similar set of APIs which accept char pointers ...

+ Reply to Thread
Results 1 to 6 of 6

Thread: memory mapped file question

  1. memory mapped file question

    Hi,
    We have some API's that accept files as one of the parameters, and
    these APIs either read or write into those files (FILE*s).
    Now we have a request to support a similar set of APIs which accept
    char pointers instead of files. In order to achieve this, we thought
    memory mapped files could be of use. But we aren't exactly sure how to
    achieve this.

    Suppose there is an API like this: int write(FILE* fp, SomeStructure*
    st);

    We need to provide : int write_s(char *s, SomeStructure* st);

    Instead of reimplementing write_s from scratch, we want to use the
    implementation of "write()". Can we do something like this?

    int write_s(char *s, SomeStructure *st) {
    fd = open("/tmp/tempfile", ...);
    ptr = mmap( ..., PROT_WRITE, fd ,...);
    write(fileno(fd), st);
    memcopy(s,ptr,..); // any way to avoid this copy?
    }

    If not, can you please suggest some better alternatives for this
    purpose?

    Thanks in advance,
    Raj

  2. Re: memory mapped file question

    Raj writes:

    > Hi,
    > We have some API's that accept files as one of the parameters, and
    > these APIs either read or write into those files (FILE*s).
    > Now we have a request to support a similar set of APIs which accept
    > char pointers instead of files. In order to achieve this, we thought
    > memory mapped files could be of use. But we aren't exactly sure how to
    > achieve this.
    >
    > Suppose there is an API like this: int write(FILE* fp, SomeStructure*
    > st);


    I'm going to call it write_file() instead to avoid confusion with write.

    > We need to provide : int write_s(char *s, SomeStructure* st);
    >
    > Instead of reimplementing write_s from scratch, we want to use the
    > implementation of "write()". Can we do something like this?
    >
    > int write_s(char *s, SomeStructure *st) {
    > fd = open("/tmp/tempfile", ...);
    > ptr = mmap( ..., PROT_WRITE, fd ,...);
    > write(fileno(fd), st);
    > memcopy(s,ptr,..); // any way to avoid this copy?
    > }


    Ugh. I suppose you could do that, but I wouldn't.

    To avoid the copy, you would have to map the file at the address pointed
    to by `s' (using the MAP_FIXED option) but it would have to be
    page-aligned for that to work. And it's ugly.

    Better would just be to read back from the temporary file:

    int write_s(char *s, SomeStructure *st) {
    FILE *f = tmpfile();
    write_file(f, st);
    rewind(f);
    fread(s, 1, SIZE, f);
    fclose(f);
    }

    This should be approximately as efficient as mmap with a copy.

    If you're on Linux or another system that uses GNU libc, you can use
    their fmemopen() feature, which gives you a FILE * that refers to a
    memory buffer:

    int write_s(char *s, SomeStructure *st) {
    FILE *f = fmemopen(s, SIZE, "w");
    write_file(f, st);
    fclose(f);
    }

    This would be faster and avoid the temporary file, but not portable.

    The cleanest thing, of course, would be to change write_file(). With
    some care you could have a single function that can write either to a
    file or a string. Or, you can rewrite it to turn it into write_s(), and
    then implement write_file() in terms of write_s(), which is considerably
    less ugly IMHO, though costs a little bit of memory:

    int write_file(FILE *f, SomeStructure *st) {
    char *s = malloc(SIZE);
    write_s(s, st);
    fwrite(s, 1, SIZE, f);
    }

    Hope this helps.

  3. Re: memory mapped file question

    > If you're on Linux or another system that uses GNU libc, you can use
    > their fmemopen() feature, which gives you a FILE * that refers to a
    > memory buffer:
    >

    This is *EXACTLY* what I need. How I wish this were available on
    Windows :-(

    Thanks a lot for your inputs, Nate!

    ~Raj
    http://www.rajorshi.net

  4. Re: memory mapped file question

    On Oct 17, 2:26*am, Raj wrote:
    > > If you're on Linux or another system that uses GNU libc, you can use
    > > their fmemopen() feature, which gives you a FILE * that refers to a
    > > memory buffer:

    >
    > This is *EXACTLY* what I need. How I wish this were available on
    > Windows :-(


    If you have MFC available, that would be the CMemFile class.

  5. Re: memory mapped file question

    On Fri, 17 Oct 2008 10:40:44 -0700, Michael Mol wrote:
    > On Oct 17, 2:26Â*am, Raj wrote:


    >>> If you're on Linux or another system that uses GNU libc, you can use
    >>> their fmemopen() feature, which gives you a FILE * that refers to a
    >>> memory buffer:

    >>
    >> This is *EXACTLY* what I need. How I wish this were available on
    >> Windows :-(

    >
    > If you have MFC available, that would be the CMemFile class.


    Mortar fire controller? I know working on windows can get irritating but
    is shelling really the best solution? :-s


  6. Re: memory mapped file question

    viza writes:

    > On Fri, 17 Oct 2008 10:40:44 -0700, Michael Mol wrote:
    >> On Oct 17, 2:26*am, Raj wrote:

    >
    >>>> If you're on Linux or another system that uses GNU libc, you can use
    >>>> their fmemopen() feature, which gives you a FILE * that refers to a
    >>>> memory buffer:
    >>>
    >>> This is *EXACTLY* what I need. How I wish this were available on
    >>> Windows :-(

    >>
    >> If you have MFC available, that would be the CMemFile class.

    >
    > Mortar fire controller? I know working on windows can get irritating but
    > is shelling really the best solution? :-s


    I'll take a Unix shell over windows anytime.

    --
    Måns Rullgård
    mans@mansr.com

+ Reply to Thread