ndbm update not visible - Unix

This is a discussion on ndbm update not visible - Unix ; Need your help in solving this ndbm problem. I have a set of two programs using an ndbm database, and using shared reader-writer lock (pthread_rwlock_t). One program is a Reader which is a persistent daemon, and the other is a ...

+ Reply to Thread
Results 1 to 10 of 10

Thread: ndbm update not visible

  1. ndbm update not visible

    Need your help in solving this ndbm problem.

    I have a set of two programs using an ndbm database,
    and using shared reader-writer lock (pthread_rwlock_t).

    One program is a Reader which is a persistent daemon,
    and the other is a Writer which is a transient program.

    What I see is that even _after_ the Writer has updated the
    value for a given key, the Reader continues to see the old value.

    I noticed that if the Reader fetches any other Key, and then
    fetches the old key, then, and only then, does it get the new value.

    The sequence is as follows:

    1. Writer: Update(Key1, Value1)

    2. Reader: Get(Key1) ---> Value1

    3. Writer: Update(Key1, Value2)

    4. Reader: Get(Key1) ---> Value1

    5. Reader: Get(KeyX) ---> ValueX

    6. Reader: Get(Key1) ---> Value2

    I was expecting that the Reader should get Value2 at step 4.

    What can I do so that the update by the Writer
    is immediately visible in the Reader?

    Thanks

  2. Re: ndbm update not visible

    "Amitava" wrote in message
    news:2a62f25a-b28e-494e-8165-776d7dd18d53@j22g2000hsf.googlegroups.com...
    > Need your help in solving this ndbm problem.
    >
    > I have a set of two programs using an ndbm database,
    > and using shared reader-writer lock (pthread_rwlock_t).
    >
    > One program is a Reader which is a persistent daemon,
    > and the other is a Writer which is a transient program.
    >
    > What I see is that even _after_ the Writer has updated the
    > value for a given key, the Reader continues to see the old value.
    >
    > I noticed that if the Reader fetches any other Key, and then
    > fetches the old key, then, and only then, does it get the new value.
    >
    > The sequence is as follows:
    >
    > 1. Writer: Update(Key1, Value1)
    >
    > 2. Reader: Get(Key1) ---> Value1
    >
    > 3. Writer: Update(Key1, Value2)
    >
    > 4. Reader: Get(Key1) ---> Value1
    >
    > 5. Reader: Get(KeyX) ---> ValueX
    >
    > 6. Reader: Get(Key1) ---> Value2
    >
    > I was expecting that the Reader should get Value2 at step 4.
    >
    > What can I do so that the update by the Writer
    > is immediately visible in the Reader?


    Are you absolutely sure that step 4 is occurring _after_ step 3? Have you
    artificially created this in a debugging environment? You could allow the
    sequence to get to step 2, and then pause the reader and writer outside of
    the critical-section. Resume the writer, complete step 3, and pause it. Then
    resume the reader. If it does not get the fresh data update then the problem
    is elsewhere. Perhaps a data-structure bug or something...


  3. Re: ndbm update not visible

    On May 30, 4:58*pm, "Chris Thomasson" wrote:
    >
    > Are you absolutely sure that step 4 is occurring _after_ step 3? Have you
    > artificially created this in a debugging environment? You could allow the
    > sequence to get to step 2, and then pause the reader and writer outside of
    > the critical-section. Resume the writer, complete step 3, and pause it. Then
    > resume the reader. If it does not get the fresh data update then the problem
    > is elsewhere. Perhaps a data-structure bug or something...- Hide quoted text -
    >
    > - Show quoted text -


    Thanks for your mail.

    Yes, I am quite confident that step 4 is _after_ step 3 and I can
    reproduce it.

    Both the programs write trace logs with time stamps.

    The Reader is actually a mail server (lmtp) which opens the dbm,
    binds to a port and listens. When it gets mail, it does a _rdlock,
    then does dbm_fetch, and then releases the lock.

    The Writer is started once in a while to update the database,
    that database dictates how the mail is to be disposed of.
    The writer requests a _wrlock, then does a dbm_store (with update)
    and then releases the lock.

    Now, I sent mail from User1 (i.e. Key1) and Reader does Value1.
    Then I run the Writer to update the database.
    The Writer also has a Query mode, and both the trace log from
    the Writer, and a subsequent Query run confirms that the
    database is indeed updated.
    I wait for 5 minutes, then send another mail from the same User1.

    I expect Reader to get Value2, but it gets Value1.

    If however, it receives mail from another user (i.e. different key)
    and then a subsequent mail from the original user,
    then everything is just fine.

    The problem appears only if the Reader makes repeated query
    with exactly the same key before and after the update.

    It looks like that the ndbm library is caching the result of the
    query,
    and if the same query is repeated, it is just skipping the actual
    lookup.

    I am thinking of changing the code to, say,
    (a) the Writer, at the end of every write, sends a signal to the
    Reader
    (b) upon receipt of the signal, the Reader should close and re-open
    the ndbm.

    The Reader uses pthreads, so its a minor hassle to handle the signal,
    but more than that, why should I need to do that at all?

    In case it's of any significance, the ndbm database has about
    a million records, the code is compiled on Solaris 9 (Sparc)
    with GCC 3.3.3 and test runs on both Sol 9 and Sol 10
    show exactly same symptom.

    I have tested the reader-writer locking mechanism separately
    by adding delays in the critical section, and the Reader / Writer
    actually waits exactly as expected.
    The pthreads man page on Solaris says that writers have priority
    to avoid starvation, and I have seen that in my prior tests.

    What am I doing wrong?
    Should I consider switching to, say, gdbm?
    Or should I close and open the database (as above)?

    Thanks

  4. Re: ndbm update not visible

    On May 30, 12:35*pm, Amitava wrote:
    > Need your help in solving this ndbm problem.
    >
    > I have a set of two programs using an ndbm database,
    > and using shared reader-writer lock (pthread_rwlock_t).

    [snip]
    > What can I do so that the update by the Writer
    > is immediately visible in the Reader?


    I don't think ndbm is designed to work this way. The lock won't help
    you, because ndbm has no idea that it's being called with a lock or
    that it doesn't have exclusive use of the database when the lock is
    not held.

    DS

  5. Re: ndbm update not visible

    >Need your help in solving this ndbm problem.
    >
    >I have a set of two programs using an ndbm database,
    >and using shared reader-writer lock (pthread_rwlock_t).


    At least one old version of 'dbm' software definitely didn't work
    in a multi-tasking-friendly manner. A customer was trying to use
    it that way, and it turned out that on opening the file, it cached
    some stuff and kept that info *forever* (there was no way to close
    the database, other than exit()).

    I'll suggest this protocol:

    1. Obtain lock (not a thread lock, a system-wide lock).
    2. Open the database.
    3. Read/write as needed.
    4. Close the database.
    5. Release lock.

    The lock may be shared if you only intend to read.



  6. Re: ndbm update not visible


    "Amitava" wrote in message
    news:411a2726-0248-45af-b92b-61b7f7037c0e@l42g2000hsc.googlegroups.com...
    On May 30, 4:58 pm, "Chris Thomasson" wrote:
    >
    > > Are you absolutely sure that step 4 is occurring _after_ step 3? Have
    > > you
    > > artificially created this in a debugging environment? You could allow
    > > the
    > > sequence to get to step 2, and then pause the reader and writer outside
    > > of
    > > the critical-section. Resume the writer, complete step 3, and pause it.
    > > Then
    > > resume the reader. If it does not get the fresh data update then the
    > > problem
    > > is elsewhere. Perhaps a data-structure bug or something...- Hide quoted
    > > text -
    > >
    > > - Show quoted text -


    [...]

    > The problem appears only if the Reader makes repeated query
    > with exactly the same key before and after the update.


    > It looks like that the ndbm library is caching the result of the
    > query,
    > and if the same query is repeated, it is just skipping the actual
    > lookup.


    > I am thinking of changing the code to, say,
    > (a) the Writer, at the end of every write, sends a signal to the
    > Reader
    > (b) upon receipt of the signal, the Reader should close and re-open
    > the ndbm.
    >
    > The Reader uses pthreads, so its a minor hassle to handle the signal,
    > but more than that, why should I need to do that at all?


    I think that's about all you can do. AFAICT, the root of the problem does
    not have anything to do with thread synchronization. Its a database issue...

    [...]


  7. Re: ndbm update not visible

    On May 30, 12:35 pm, Amitava wrote:
    > Need your help in solving this ndbm problem.
    >

    Sorry for the transgression....

    I am interested in using a file based database. Essentially is will
    hold about 1GB of data and be accessed for fast lookups by multiple
    processes (from C code) on a single box. Occasionally it will need to
    be locked for updates.

    I am assuming ndbm is a possible choice for this, as is berkeley db.
    Can you recommend what are the major options for these types of
    systems and which one(s) is the best supported?

    Thanks,
    Ivan Novick

  8. Re: ndbm update not visible

    Thanks to all for your replies, esp, David and Chris.

    So it's a _feature_ of ndbm and I'll have to factor that into my
    solution.

    My situation is similar to Ivan's. My Reader is a set of daemon
    processes and the Writer is a transient process.

    In my case, the Writer runs less than 10 times per hour,
    whereas the lookups maybe as high as 100K per hour,
    so the signalling solution shouldn't have any visible impact.

    I haven't used Berkeley DB, but I understand it has a lot more
    features, e.g. cursors. OTOH, many issues are being discussed
    on the Oracle BDB forums.

    For simple lookups, I think ndbm is a good choice.

    Thanks

  9. Re: ndbm update not visible

    On Jun 1, 7:03*am, Amitava wrote:

    > In my case, the Writer runs less than 10 times per hour,
    > whereas the lookups maybe as high as 100K per hour,
    > so the signalling solution shouldn't have any visible impact.


    How about this:

    The writer writes the new data to a new file and then renames the new
    file on top of the old file. The readers keep track of the
    modification timestamp of the file. Every minute or so, they check if
    the modification timestamp has changed, and if so they re-read the
    data.

    DS

  10. Re: ndbm update not visible

    On Jun 2, 12:01*am, David Schwartz wrote:
    >
    > How about this:
    >
    > The writer writes the new data to a new file and then renames the new
    > file on top of the old file. The readers keep track of the
    > modification timestamp of the file. Every minute or so, they check if
    > the modification timestamp has changed, and if so they re-read the
    > data.
    >
    > DS



    Thanks !

    The present version, in production for about 4 years, uses (almost)
    this
    technique. The difference is, the Readers are not daemons but
    transient
    processes so the new ones automatically "see" the new database.

    [ the old ones read from stdin while the new one talks over sockets
    but that's not important, they do use the same database lookups
    though ]

    However, the creation of the new database (actually copying and
    updating)
    is a resource hog; on the red hot Sun Fire 880, it takes 90+ seconds ;-
    (.

    Avoiding that hog is an objective of the present project ;-}

    I've been testing the signalling from the Writer and re-opening
    the db and that solves the original problem.

    Thanks for all the help.

+ Reply to Thread