large file access at different locations - Linux
This is a discussion on large file access at different locations - Linux ; Hi
Although this is not a "system" question, but people here most probably have
the knowledge for an answer.
I have an applicaton which writes to a large file in a circular manner and
reads from different locations. There are ...
-
large file access at different locations
Hi
Although this is not a "system" question, but people here most probably have
the knowledge for an answer.
I have an applicaton which writes to a large file in a circular manner and
reads from different locations. There are at least two threads, one is
writing others are reading. The writer it writing sequential, the readers
may read seqential but also random. Read and write positions may differ
drastically, access is serialized with mutexes and the file handle(es) are
always open.
Now the question:
Is it of any performance benefit to use different filehandles for each
accessor and why?
Is it of any performance benefit to use different filehandles with memory
mapped I/O for each accessor and why?
Are there any pitfalls even if access is serialized (only one handle is
active at a time) ?
Thanks,
Ariovist
-
Re: large file access at different locations
Ariovist Lowa wrote:
> I have an applicaton which writes to a large file in a circular manner and
> reads from different locations. There are at least two threads, one is
> writing others are reading. The writer it writing sequential, the readers
> may read seqential but also random. Read and write positions may differ
> drastically, access is serialized with mutexes and the file handle(es) are
> always open.
This doesn't directly address your question, but have you considered
memory mapping the file? You could potentially use fcntl()
reader/writer locking to lock specific areas of the file to improve
concurrency if that's an issue.
Chris
-
Re: large file access at different locations
Ariovist Lowa wrote:
> I have an applicaton which writes to a large file in a circular manner and
> reads from different locations. There are at least two threads, one is
> writing others are reading. The writer it writing sequential, the readers
> may read seqential but also random. Read and write positions may differ
> drastically, access is serialized with mutexes and the file handle(es) are
> always open.
What is the purpose of the mutex? Do you have a platform that doesn't
have 'pread'/'pwrite'? If so, you probably want to open the file twice,
so that you can continue to perform writes while you're waiting for a
read to complete.
DS
