Files over 2GB in C++ on RHEL3 - Linux
This is a discussion on Files over 2GB in C++ on RHEL3 - Linux ; I am using Red Hat Enterprise Linux 3 and trying to open files of 2GB
or greater using the following example syntax.
#include
#include
#include
#include
int main ( int argc , char** argv )
{
std::fstream inFile;
inFile.open ( ...
-
Files over 2GB in C++ on RHEL3
I am using Red Hat Enterprise Linux 3 and trying to open files of 2GB
or greater using the following example syntax.
#include
#include
#include
#include
int main ( int argc , char** argv )
{
std::fstream inFile;
inFile.open ( "./2GB_file.txt" , std::ios::in|std::ios::binary );
if ( inFile.good ( ) )
{
std::cout << "File opened OK" << std::endl;
}
else
{
std::cout << "File failed to open" << std::endl;
std::cout << "Error is " << errno << " : " << strerror ( errno )
<< "." << std::endl;
}
inFile.close ( );
}
I compile with large file support (flags -D_FILE_OFFSET_BITS=64
-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE)
However the files still fail to open, giving the error
File failed to open
Error is 27 : File too large.
Has anyone come across this problem and found a solution. The old C
file calls (fopen) are able to open a file greater than 2GB. The
problem appears specific to C++.
Jon.
-
Re: Files over 2GB in C++ on RHEL3
Hello,
jcombe@acxiom.co.uk wrote:
> I am using Red Hat Enterprise Linux 3 and trying to open files of 2GB
> or greater using the following example syntax.
>
> std::fstream inFile;
> inFile.open ( "./2GB_file.txt" , std::ios::in|std::ios::binary );
> File failed to open
> Error is 27 : File too large.
>
> Has anyone come across this problem and found a solution. The old C
> file calls (fopen) are able to open a file greater than 2GB. The
> problem appears specific to C++.
Since the iostream-library is meant to do buffering on its own, it has
to maintain file offsets on its own, which needs more than 32 bits for
large file support. I can reproduce the problem using gcc-3.3. With
gcc-3.4 large files seem to be supported. Since the patch is probably
not small, and gcc-3.3 has been closed for changes for some time, you
have the choice of using a newer the compiler or using another STL,
e.g. STLport might be worth a try (I don't know for sure, whether it
supports large files). Both will break binary compatibility, RHEL3
using gcc-3.2.3, AFAIK. YOu could try to look at the streambuf classes
of newer gcc releases to get an idea howto do large file support on
your own. This is probably what I would try, if I had to make something
run on the old system.
Bernd Strieder
-
Re: Files over 2GB in C++ on RHEL3
> Since the iostream-library is meant to do buffering on its own, it has
> to maintain file offsets on its own, which needs more than 32 bits for
> large file support. I can reproduce the problem using gcc-3.3. With
> gcc-3.4 large files seem to be supported. Since the patch is probably
> not small, and gcc-3.3 has been closed for changes for some time, you
> have the choice of using a newer the compiler or using another STL,
> e.g. STLport might be worth a try (I don't know for sure, whether it
> supports large files). Both will break binary compatibility, RHEL3
> using gcc-3.2.3, AFAIK. YOu could try to look at the streambuf classes
> of newer gcc releases to get an idea howto do large file support on
> your own. This is probably what I would try, if I had to make something
> run on the old system.
Bernd,
Thanks very much for the reply. I've since found the following which
confirms it as a bug, and confirms that it won't be fixed in RHEL3 for
the reason you mention - binary compatability.
https://bugzilla.redhat.com/bugzilla....cgi?id=117008
So the solutions looks to be to try one of the options you have
suggested (STLport) or re-write the code using C (fopen, fread etc).
Jon.
