getgrgid(): multiple entries with 1 GID - Unix

This is a discussion on getgrgid(): multiple entries with 1 GID - Unix ; Hi all; There's a long-standing history of every UNIX I've ever used that if you have multiple group entries with the same GID, running getgrgid() for that GID is guaranteed to always return you the first matching entry. This holds ...

+ Reply to Thread
Results 1 to 8 of 8

Thread: getgrgid(): multiple entries with 1 GID

  1. getgrgid(): multiple entries with 1 GID

    Hi all;

    There's a long-standing history of every UNIX I've ever used that if you
    have multiple group entries with the same GID, running getgrgid() for
    that GID is guaranteed to always return you the first matching entry.
    This holds true regardless of whether the group entries are in
    /etc/group, or in NIS.

    I'm hoping someone can point me to some "official" documentation of
    this feature. Obviously the POSIX standard doesn't really deal with
    group files or NIS so there's nothing there; is this documented in any
    per-OS (Solaris, HP-UX, etc.) docs? Many enterprise sites use the
    feature to get around the 1024 character limit in group lists, by simply
    creating multiple group entries with different names but the same GID.
    I need to show that this behavior is not just a fluke, but is actually a
    feature.

    I'm seeing that the Linux nscd process (at least, I think it's nscd) is
    sometimes getting into a funky state where it returns a different entry
    when looking up the same GID. Apparently saying that it's worked that
    way in every UNIX I've ever used is not good enough for them :-).

    --
    -------------------------------------------------------------------------------
    Paul D. Smith HASMAT--HA Software Mthds & Tools
    "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
    -------------------------------------------------------------------------------
    These are my opinions--Nortel takes no responsibility for them.

  2. Re: getgrgid(): multiple entries with 1 GID

    "Paul D. Smith" writes:

    >There's a long-standing history of every UNIX I've ever used that if you
    >have multiple group entries with the same GID, running getgrgid() for
    >that GID is guaranteed to always return you the first matching entry.
    >This holds true regardless of whether the group entries are in
    >/etc/group, or in NIS.


    For NIS this was never true and the match could be random
    (depends somewhat on the underlying database).

    >I'm seeing that the Linux nscd process (at least, I think it's nscd) is
    >sometimes getting into a funky state where it returns a different entry
    >when looking up the same GID. Apparently saying that it's worked that
    >way in every UNIX I've ever used is not good enough for them :-).


    For /etc/group, that's inexcusable; for another nameservice, well,
    whatever it does, is what the daemon should follow.

    Casper
    --
    Expressed in this posting are my opinions. They are in no way related
    to opinions held by my employer, Sun Microsystems.
    Statements on Sun products included here are not gospel and may
    be fiction rather than truth.

  3. Re: getgrgid(): multiple entries with 1 GID

    %% Casper H.S. Dik writes:

    chsd> "Paul D. Smith" writes:
    >> There's a long-standing history of every UNIX I've ever used that
    >> if you have multiple group entries with the same GID, running
    >> getgrgid() for that GID is guaranteed to always return you the
    >> first matching entry. This holds true regardless of whether the
    >> group entries are in /etc/group, or in NIS.


    chsd> For NIS this was never true and the match could be random
    chsd> (depends somewhat on the underlying database).

    I'm talking about the traditional DBM-based NIS databases. My
    understanding is that the makedbm script to generate NIS databases will
    walk through the group definitions in order, and if it gets to a second
    entry that conflicts with the first one, the second is dropped.

    That would mean that in the group.bygid NIS database, only the first GID
    entry would appear and the others would be thrown out.

    Thus, every getgrgid() request would always and was guaranteed to return
    the data associated with the first entry in the group file.


    There are many enterprises which have used this fact to create very
    large (>1024 characters) group lists, on SunOS4, Solaris, HP-UX, AIX,
    and DG-UX that I'm immediately familiar with, so I would be shocked to
    learn that all of them were relying so heavily on an arbitrary
    capability.

    --
    -------------------------------------------------------------------------------
    Paul D. Smith HASMAT--HA Software Mthds & Tools
    "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
    -------------------------------------------------------------------------------
    These are my opinions--Nortel takes no responsibility for them.

  4. Re: getgrgid(): multiple entries with 1 GID

    >There's a long-standing history of every UNIX I've ever used that if you
    >have multiple group entries with the same GID, running getgrgid() for
    >that GID is guaranteed to always return you the first matching entry.


    I worked with old Unices long ago (Targon/32, Altos). They had some
    weird code for handling getpwuid getgrgid ... and maintained sometimes
    hashes. At least getpwuid on Altos did not have the described property
    (I made several admin entries with uid=0 for different admins and
    passwords, and the ls -l did not show root for uid=0 any more, although
    I put the others after root).

    Hubble.


  5. Re: getgrgid(): multiple entries with 1 GID

    gordonb.yucjl@burditt.org (Gordon Burditt) writes:

    >>I'm talking about the traditional DBM-based NIS databases. My
    >>understanding is that the makedbm script to generate NIS databases will
    >>walk through the group definitions in order, and if it gets to a second
    >>entry that conflicts with the first one, the second is dropped.


    >DBM, at least, can consistently deal with accepting the first one
    >or the last one. If you pass the DBM_INSERT flag to dbm_store(),
    >the first insert works and subsequent ones get an error (which the
    >code ignores. The "duplicate key" error is distinguishable from
    >other errors). If you pass the DBM_REPLACE flag, all of them succeed
    >and you end up with the last one in the DBM file.


    The dbm_store calls in Solaris makedbm all use DBM_REPLACE.

    Casper
    --
    Expressed in this posting are my opinions. They are in no way related
    to opinions held by my employer, Sun Microsystems.
    Statements on Sun products included here are not gospel and may
    be fiction rather than truth.

  6. Re: getgrgid(): multiple entries with 1 GID

    >I'm talking about the traditional DBM-based NIS databases. My
    >understanding is that the makedbm script to generate NIS databases will
    >walk through the group definitions in order, and if it gets to a second
    >entry that conflicts with the first one, the second is dropped.


    DBM, at least, can consistently deal with accepting the first one
    or the last one. If you pass the DBM_INSERT flag to dbm_store(),
    the first insert works and subsequent ones get an error (which the
    code ignores. The "duplicate key" error is distinguishable from
    other errors). If you pass the DBM_REPLACE flag, all of them succeed
    and you end up with the last one in the DBM file.

    >That would mean that in the group.bygid NIS database, only the first GID
    >entry would appear and the others would be thrown out.
    >
    >Thus, every getgrgid() request would always and was guaranteed to return
    >the data associated with the first entry in the group file.


    You can easily write DBM code to do this. Whether or not people
    have consistently written the code this way for this application
    is unclear.


    >There are many enterprises which have used this fact to create very
    >large (>1024 characters) group lists, on SunOS4, Solaris, HP-UX, AIX,
    >and DG-UX that I'm immediately familiar with, so I would be shocked to
    >learn that all of them were relying so heavily on an arbitrary
    >capability.


    Gordon L. Burditt

  7. Re: getgrgid(): multiple entries with 1 GID

    %% Casper H.S. Dik writes:

    chsd> The dbm_store calls in Solaris makedbm all use DBM_REPLACE.

    Strange; I was sure we always made the entry we wanted first. But maybe
    we make it last. Or maybe we use a different makedbm. Whatever method
    we use for this, we've been using for 12+ years on Sun systems and it's
    always worked perfectly, as expected.

    Are you saying that Sun (and other UNIX systems) simply don't support
    groups with membership lists >1024 characters long in NIS w/ DBM? I
    believe this method of supporting longer lists of users is very common
    and is implicitly, if not explicitly, supported by every UNIX vendor.


    Anyway, I've debugged this further. When the problem happens, I can run
    ypcat -k group.bygid and see that the GID in question (say 5000)
    resolves to exactly one entry (as you'd expect) with the correct group
    name (say "foobar").

    But then I run perl -e '$n = getgrgid(5000); print "$n\n";' and it
    prints "foobarX001" and not "foobar". Using strace shows it's getting
    this info from the nscd socket.

    To me this clearly shows a bug, probably in in nscd's caching.
    Comments?

    Restarting NSCD solves the problem, but it comes back again. We've
    resorted to installing a cron job that runs every few minutes, checks for
    this issue, and restarts NSCD if necessary.


    Now, it's true that if I run ypcat -k group.byname I DO get an entry
    "foobar" with GID 1563 and another entry "foobarX001" also with GID
    1563, but that's not illegal as far as I'm aware since the keys are
    different.

    --
    -------------------------------------------------------------------------------
    Paul D. Smith HASMAT--HA Software Mthds & Tools
    "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
    -------------------------------------------------------------------------------
    These are my opinions--Nortel takes no responsibility for them.

  8. Re: getgrgid(): multiple entries with 1 GID

    "Paul D. Smith" writes:

    >%% Casper H.S. Dik writes:


    > chsd> The dbm_store calls in Solaris makedbm all use DBM_REPLACE.


    >Strange; I was sure we always made the entry we wanted first. But maybe
    >we make it last. Or maybe we use a different makedbm. Whatever method
    >we use for this, we've been using for 12+ years on Sun systems and it's
    >always worked perfectly, as expected.


    The way to support this was always to have multiple entries and
    that gets you multiple groups. THe mapping back to the name
    may not always function properly.

    >Anyway, I've debugged this further. When the problem happens, I can run
    >ypcat -k group.bygid and see that the GID in question (say 5000)
    >resolves to exactly one entry (as you'd expect) with the correct group
    >name (say "foobar").


    >But then I run perl -e '$n = getgrgid(5000); print "$n\n";' and it
    >prints "foobarX001" and not "foobar". Using strace shows it's getting
    >this info from the nscd socket.


    That doesn't look good; perhaps nscd does something like:

    grp = getgrnam(....)

    if (!findhash(grp->gr_gid))
    enterhash(gidhash, grp->gr_gid, grp);

    Casper
    --
    Expressed in this posting are my opinions. They are in no way related
    to opinions held by my employer, Sun Microsystems.
    Statements on Sun products included here are not gospel and may
    be fiction rather than truth.

+ Reply to Thread