Files over 2GB in C++ on RHEL3 - Linux

This is a discussion on Files over 2GB in C++ on RHEL3 - Linux ; I am using Red Hat Enterprise Linux 3 and trying to open files of 2GB or greater using the following example syntax. #include #include #include #include int main ( int argc , char** argv ) { std::fstream inFile; inFile.open ( ...

+ Reply to Thread
Results 1 to 3 of 3

Thread: Files over 2GB in C++ on RHEL3

  1. Files over 2GB in C++ on RHEL3

    I am using Red Hat Enterprise Linux 3 and trying to open files of 2GB
    or greater using the following example syntax.

    #include
    #include
    #include
    #include

    int main ( int argc , char** argv )
    {
    std::fstream inFile;
    inFile.open ( "./2GB_file.txt" , std::ios::in|std::ios::binary );

    if ( inFile.good ( ) )
    {
    std::cout << "File opened OK" << std::endl;
    }
    else
    {
    std::cout << "File failed to open" << std::endl;
    std::cout << "Error is " << errno << " : " << strerror ( errno )
    << "." << std::endl;
    }

    inFile.close ( );
    }

    I compile with large file support (flags -D_FILE_OFFSET_BITS=64
    -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE)

    However the files still fail to open, giving the error

    File failed to open
    Error is 27 : File too large.

    Has anyone come across this problem and found a solution. The old C
    file calls (fopen) are able to open a file greater than 2GB. The
    problem appears specific to C++.

    Jon.


  2. Re: Files over 2GB in C++ on RHEL3

    Hello,

    jcombe@acxiom.co.uk wrote:

    > I am using Red Hat Enterprise Linux 3 and trying to open files of 2GB
    > or greater using the following example syntax.
    >


    > std::fstream inFile;
    > inFile.open ( "./2GB_file.txt" , std::ios::in|std::ios::binary );


    > File failed to open
    > Error is 27 : File too large.
    >
    > Has anyone come across this problem and found a solution. The old C
    > file calls (fopen) are able to open a file greater than 2GB. The
    > problem appears specific to C++.


    Since the iostream-library is meant to do buffering on its own, it has
    to maintain file offsets on its own, which needs more than 32 bits for
    large file support. I can reproduce the problem using gcc-3.3. With
    gcc-3.4 large files seem to be supported. Since the patch is probably
    not small, and gcc-3.3 has been closed for changes for some time, you
    have the choice of using a newer the compiler or using another STL,
    e.g. STLport might be worth a try (I don't know for sure, whether it
    supports large files). Both will break binary compatibility, RHEL3
    using gcc-3.2.3, AFAIK. YOu could try to look at the streambuf classes
    of newer gcc releases to get an idea howto do large file support on
    your own. This is probably what I would try, if I had to make something
    run on the old system.

    Bernd Strieder


  3. Re: Files over 2GB in C++ on RHEL3

    > Since the iostream-library is meant to do buffering on its own, it has
    > to maintain file offsets on its own, which needs more than 32 bits for
    > large file support. I can reproduce the problem using gcc-3.3. With
    > gcc-3.4 large files seem to be supported. Since the patch is probably
    > not small, and gcc-3.3 has been closed for changes for some time, you
    > have the choice of using a newer the compiler or using another STL,
    > e.g. STLport might be worth a try (I don't know for sure, whether it
    > supports large files). Both will break binary compatibility, RHEL3
    > using gcc-3.2.3, AFAIK. YOu could try to look at the streambuf classes
    > of newer gcc releases to get an idea howto do large file support on
    > your own. This is probably what I would try, if I had to make something
    > run on the old system.


    Bernd,

    Thanks very much for the reply. I've since found the following which
    confirms it as a bug, and confirms that it won't be fixed in RHEL3 for
    the reason you mention - binary compatability.

    https://bugzilla.redhat.com/bugzilla....cgi?id=117008

    So the solutions looks to be to try one of the options you have
    suggested (STLport) or re-write the code using C (fopen, fread etc).

    Jon.


+ Reply to Thread