star, extended attributes, UID and GID ... - OS2

This is a discussion on star, extended attributes, UID and GID ... - OS2 ; Hello, I recently took a look at 'star' for backup purposes. After restoring a test-backup, all files which previously had no EAs, showed 124 byte EAs with UID, GID, etc. The files which initially had EAs were restored without additional ...

+ Reply to Thread
Results 1 to 14 of 14

Thread: star, extended attributes, UID and GID ...

  1. star, extended attributes, UID and GID ...

    Hello,

    I recently took a look at 'star' for backup purposes. After restoring a test-backup, all files which previously had no EAs,
    showed 124 byte EAs with UID, GID, etc. The files which initially had EAs were restored without additional UID, GID, etc.
    Looks strange to me.

    Since I don't want to clutter the filesystem with useless EAs, I'd like to switch this off. Only the original EAs should be restored.
    Any way to do this?

    Is this caused by star itself or the used c-library?

    Thanks and regards
    Stefan




  2. Re: star, extended attributes, UID and GID ...

    Stefan Pelz wrote:
    > Hello,
    >
    > I recently took a look at 'star' for backup purposes. After restoring a test-backup, all files which previously had no EAs,
    > showed 124 byte EAs with UID, GID, etc. The files which initially had EAs were restored without additional UID, GID, etc.
    > Looks strange to me.
    >
    > Since I don't want to clutter the filesystem with useless EAs, I'd like to switch this off. Only the original EAs should be restored.
    > Any way to do this?
    >
    > Is this caused by star itself or the used c-library?
    >

    It might be the -acl switch. If you are using the star_rotation.cmd you
    could try removing it. (or adding it if you don't have it now).
    You could also try using the -nochown flag or the -no-p flag.
    Its not to bad having the EAs as OS/2 (at least on HPFS) allocates 512
    bytes for EAs for every file anyways.
    It is probably an interaction between Star and the c-library. Star due
    to *nix roots wants to use UID etc and the c-library allows it. But I
    really don't know.
    Dave

  3. Re: star, extended attributes, UID and GID ...

    Stefan Pelz wrote:
    > I recently took a look at 'star' for backup purposes. After restoring a test-backup, all files which previously had no EAs,
    > showed 124 byte EAs with UID, GID, etc. The files which initially had EAs were restored without additional UID, GID, etc.
    > Looks strange to me.
    >
    > Since I don't want to clutter the filesystem with useless EAs, I'd like to switch this off. Only the original EAs should be restored.
    > Any way to do this?
    >
    > Is this caused by star itself or the used c-library?


    The latter. As far as I know newer libc versions (since libc06 or so)
    create these EAs to emulate some behaviour of unix systems.
    However, it will not harm you unless you are using a FAT filesystem.
    HPFS stores small EAs like these in the FNode. You will not have
    noticable overhead from the EAs.


    Marcel

  4. Re: star, extended attributes, UID and GID ...

    On Tue, 14 Aug 2007 19:43:53 UTC, Marcel Müller wrote:
    > Stefan Pelz wrote:


    > > I recently took a look at 'star' for backup purposes. After restoring a test-backup,
    > > all files which previously had no EAs, showed 124 byte EAs with UID, GID, etc. The
    > > files which initially had EAs were restored without additional UID, GID, etc.
    > > Looks strange to me.
    > >
    > > Since I don't want to clutter the filesystem with useless EAs, I'd like to switch this
    > > off. Only the original EAs should be restored. Any way to do this?
    > >
    > > Is this caused by star itself or the used c-library?

    >
    > The latter. As far as I know newer libc versions (since libc06 or so)
    > create these EAs to emulate some behaviour of unix systems.
    > However, it will not harm you unless you are using a FAT filesystem.
    > HPFS stores small EAs like these in the FNode. You will not have
    > noticable overhead from the EAs.


    I find stuff like this to be somewhere between annoying & offensive.
    It kills me that some developers, even those who've made valuable
    contributions like Knut St O., seem to think that their "features"
    are so terribly important that they have to be imposed on everyone.
    Can anyone name a single app that actually _needs_ this stuff?

    As to "no noticable overhead", I have to disagree. Whenever you
    open a WPS folder, the WPS checks each file for .TYPE, .LONGNAME,
    and .CLASSINFO EAs. Since most files have no EAs, the process is
    quick. Now, however, it has to fetch the EAs for every single file
    and parse them, simply to discover that there isn't anything useful
    there.

    Another consideration: one source (the only one I could find) says
    that HPFS leaves 145 bytes available in an FNode for EAs. This
    should be enough space to hold the .TYPE and .LONGNAME EAs for most
    files. But if these extra EAs are added first, then the useful EAs
    will be forced to to another sector, possibly requiring read-head
    movement to access them. This is hardly "low overhead".


    --
    == == almost usable email address: rws AT e-vertise.com == ==
    __________________________________________________ _________________
    |
    | Remote Workplace Server v0.80
    Rich Walsh | interact with the WPS from any program
    Ft Myers, FL | http://e-vertise.com/rws/rws080.zip
    __________________________________________________ _________________

  5. Re: star, extended attributes, UID and GID ...

    Rich Walsh wrote:
    > On Tue, 14 Aug 2007 19:43:53 UTC, Marcel Müller wrote:
    >> Stefan Pelz wrote:

    >
    >>> I recently took a look at 'star' for backup purposes. After restoring a test-backup,
    >>> all files which previously had no EAs, showed 124 byte EAs with UID, GID, etc. The
    >>> files which initially had EAs were restored without additional UID, GID, etc.
    >>> Looks strange to me.
    >>>
    >>> Since I don't want to clutter the filesystem with useless EAs, I'd like to switch this
    >>> off. Only the original EAs should be restored. Any way to do this?
    >>>
    >>> Is this caused by star itself or the used c-library?

    >> The latter. As far as I know newer libc versions (since libc06 or so)
    >> create these EAs to emulate some behaviour of unix systems.
    >> However, it will not harm you unless you are using a FAT filesystem.
    >> HPFS stores small EAs like these in the FNode. You will not have
    >> noticable overhead from the EAs.

    >
    > I find stuff like this to be somewhere between annoying & offensive.
    > It kills me that some developers, even those who've made valuable
    > contributions like Knut St O., seem to think that their "features"
    > are so terribly important that they have to be imposed on everyone.
    > Can anyone name a single app that actually _needs_ this stuff?
    >


    Interestingly, looking at a build tree I was just compiling. The
    executable contains 2 EAs. APPTYPE:LP Binary and TYPE: Executable. But a
    copy created with cp.exe compiled with libc063 has all the GID, UID etc.
    So while Libc enables the possibility of having this stuff it is the
    tools that create them. This includes GCC, LD etc. So in the case of the
    original poster it would be Star that is adding them.
    Really it should work the same as Linux (which does use them on HPFS)
    where they are only written if UID etc are not 0 (root)

    > As to "no noticable overhead", I have to disagree. Whenever you
    > open a WPS folder, the WPS checks each file for .TYPE, .LONGNAME,
    > and .CLASSINFO EAs. Since most files have no EAs, the process is
    > quick. Now, however, it has to fetch the EAs for every single file
    > and parse them, simply to discover that there isn't anything useful
    > there.
    >
    > Another consideration: one source (the only one I could find) says
    > that HPFS leaves 145 bytes available in an FNode for EAs. This
    > should be enough space to hold the .TYPE and .LONGNAME EAs for most
    > files. But if these extra EAs are added first, then the useful EAs
    > will be forced to to another sector, possibly requiring read-head
    > movement to access them. This is hardly "low overhead".
    >
    >

    I always understood that HPFS allocates 512 bytes for EAs but have no
    sources and could well be wrong.
    Dave

  6. Re: star, extended attributes, UID and GID ...

    Dave Yeo wrote:

    > Interestingly, looking at a build tree I was just compiling. The
    > executable contains 2 EAs. APPTYPE:LP Binary and TYPE: Executable. But a
    > copy created with cp.exe compiled with libc063 has all the GID, UID etc.
    > So while Libc enables the possibility of having this stuff it is the
    > tools that create them. This includes GCC, LD etc. So in the case of the
    > original poster it would be Star that is adding them.
    > Really it should work the same as Linux (which does use them on HPFS)
    > where they are only written if UID etc are not 0 (root)
    >

    My mistake, the executable was built with ilink.exe, so of course it has
    different EAs
    Dave

  7. Re: star, extended attributes, UID and GID ...

    On Wed, 15 Aug 2007 23:32:36 UTC, Dave Yeo wrote:

    > Interestingly, looking at a build tree I was just compiling. The
    > executable contains 2 EAs. APPTYPE:LP Binary and TYPE: Executable. But a
    > copy created with cp.exe compiled with libc063 has all the GID, UID etc.
    > So while Libc enables the possibility of having this stuff it is the
    > tools that create them. This includes GCC, LD etc. So in the case of the
    > original poster it would be Star that is adding them.
    > Really it should work the same as Linux (which does use them on HPFS)
    > where they are only written if UID etc are not 0 (root)


    I recently got a private build of Firefox from Peter & discovered that
    every single file in the package had these EAs, including things like
    ..GIFs that never got anywhere near GCC. Since the build process uses
    'cp' to construct the final product, it sounds like that's the culprit.

    FWIW, I wrote to Peter to tell him about the EAs (and to rant a bit :-)
    I also suggested he add '-X' to his zip commandline. His next build was
    EA-free.


    --
    == == almost usable email address: rws AT e-vertise.com == ==
    __________________________________________________ _________________
    |
    | Remote Workplace Server v0.80
    Rich Walsh | interact with the WPS from any program
    Ft Myers, FL | http://e-vertise.com/rws/rws080.zip
    __________________________________________________ _________________

  8. Re: star, extended attributes, UID and GID ...

    On 15 Aug., 23:49, "Rich Walsh" wrote:
    > I find stuff like this to be somewhere between annoying & offensive.
    > It kills me that some developers, even those who've made valuable
    > contributions like Knut St O., seem to think that their "features"
    > are so terribly important that they have to be imposed on everyone.
    > Can anyone name a single app that actually _needs_ this stuff?


    I totally agree. At least there should be a way to switch this off.

    > As to "no noticable overhead", I have to disagree. Whenever you
    > open a WPS folder, the WPS checks each file for .TYPE, .LONGNAME,
    > and .CLASSINFO EAs. Since most files have no EAs, the process is
    > quick. Now, however, it has to fetch the EAs for every single file
    > and parse them, simply to discover that there isn't anything useful
    > there.
    >
    > Another consideration: one source (the only one I could find) says
    > that HPFS leaves 145 bytes available in an FNode for EAs. This
    > should be enough space to hold the .TYPE and .LONGNAME EAs for most
    > files. But if these extra EAs are added first, then the useful EAs
    > will be forced to to another sector, possibly requiring read-head
    > movement to access them. This is hardly "low overhead".


    In case of 'star' this also means that after restoring a backup, each
    and every file will have the EAs. On the next backup these EAs will go
    into the backup, causing and overhead there.

    Unless 'star' takes recognizes that these are the UID, GID, ... EAs
    and puts them in the file header only. But I doubt it works this way.
    After restoring a file with other EAs, the UID, GID, ... EAs are not
    there. I assume 'star' first creates the file and afterwards replaces
    the EAs with those from the archive. Doesn't look like a complete
    implementation of the UID, GID, ... feature.

    In the past, I've spend some time to get rid of most of the unnecessay
    EAs on my drives, and I really don't want to see them reappear. Unless
    one of the proposed switches disables this behaviour, this is a show-
    stopper.

    Regards
    Stefan


  9. Re: star, extended attributes, UID and GID ...

    On Mon, 13 Aug 2007 15:48:17 GMT, Dave Yeo wrote:

    >It might be the -acl switch. If you are using the star_rotation.cmd you
    >could try removing it. (or adding it if you don't have it now).
    >You could also try using the -nochown flag or the -no-p flag.


    Does not help :-(

    Regards
    Stefan



  10. Re: star, extended attributes, UID and GID ...

    Stefan Pelz wrote:
    > On Mon, 13 Aug 2007 15:48:17 GMT, Dave Yeo wrote:
    >
    >> It might be the -acl switch. If you are using the star_rotation.cmd you
    >> could try removing it. (or adding it if you don't have it now).
    >> You could also try using the -nochown flag or the -no-p flag.

    >
    > Does not help :-(
    >
    > Regards
    > Stefan
    >
    >

    Wonder how hard it would be to recompile with GCC 3.2.2? If I get a
    chance I'll try.
    Dave

  11. Re: star, extended attributes, UID and GID ...

    Dave Yeo wrote:
    > Stefan Pelz wrote:
    >> On Mon, 13 Aug 2007 15:48:17 GMT, Dave Yeo wrote:
    >>
    >>> It might be the -acl switch. If you are using the star_rotation.cmd
    >>> you could try removing it. (or adding it if you don't have it now).
    >>> You could also try using the -nochown flag or the -no-p flag.

    >>
    >> Does not help :-(
    >>
    >> Regards
    >> Stefan
    >>
    >>

    > Wonder how hard it would be to recompile with GCC 3.2.2? If I get a
    > chance I'll try.
    > Dave


    It is not happy about the lack of iconv functionality and I remembered
    that 3.2.2 does not have large file support IIRC. Anyways don't have
    time to play with it
    Dave

  12. Re: star, extended attributes, UID and GID ...

    Hi,

    > there. I assume 'star' first creates the file and afterwards replaces
    > the EAs with those from the archive. Doesn't look like a complete
    > implementation of the UID, GID, ... feature.


    right. UID/GID are a new libc feature, I'll see if I can drop the
    creation of such attributes. libc creates such attributes when file is
    created/closed, then star rewrites EA if they were in the backup file.


    --
    Bye,

    Yuri Dario

    /*
    * member of TeamOS/2 - Italy
    * http://www.os2power.com/yuri
    * http://www.teamos2.it
    */


  13. Re: star, extended attributes, UID and GID ...

    Rich Walsh wrote:
    > As to "no noticable overhead", I have to disagree. Whenever you
    > open a WPS folder, the WPS checks each file for .TYPE, .LONGNAME,
    > and .CLASSINFO EAs. Since most files have no EAs, the process is
    > quick. Now, however, it has to fetch the EAs for every single file
    > and parse them, simply to discover that there isn't anything useful
    > there.


    That's not the way the WPS works.
    The WPS does not scan for the files and access them a second time at
    all. It issues a DosFindFirst command with info level QUERYEASFROMLIST
    to the filesystem driver and expicitly queries the EAs
    ..ICON
    ..TYPE
    ..APPTYPE
    ..CHECKSUM
    ..ASSOCTABLE
    ..LONGNAME
    ..CLASSINFO
    ..PREVCLASS
    and nothing else. This unusual command is the reason for many problems
    with Samba 3 up to release 3.0.16 or so. It is also the reason for the
    relatively low filesystem activity caused by the WPS.


    > Another consideration: one source (the only one I could find) says
    > that HPFS leaves 145 bytes available in an FNode for EAs.


    316 bytes.


    > This
    > should be enough space to hold the .TYPE and .LONGNAME EAs for most
    > files. But if these extra EAs are added first, then the useful EAs
    > will be forced to to another sector, possibly requiring read-head
    > movement to access them. This is hardly "low overhead".


    The .CLASSINFO from the WPS is much more likely to overflow the EA space
    in the FNode. .ICON does it anyway, if associated.
    Of course there is a low chance that the few additional EAs cause a
    sector allocation. But I do not claim about that since there are
    thousands of other things lying around on the harddisk of any OS/2
    installation and never used. These things are much bigger than anything
    like this could ever grow.


    Marcel

  14. Re: star, extended attributes, UID and GID ...

    stefan.pelz@t-online.de wrote:
    > Unless 'star' takes recognizes that these are the UID, GID, ... EAs
    > and puts them in the file header only. But I doubt it works this way.
    > After restoring a file with other EAs, the UID, GID, ... EAs are not
    > there. I assume 'star' first creates the file and afterwards replaces
    > the EAs with those from the archive. Doesn't look like a complete
    > implementation of the UID, GID, ... feature.


    You are right, but if the proxy implementation in clib is symmetric it
    will not return these EAs twice. Once in stat and once as EAs. Any other
    implementation makes no sense in my opinion.
    So if you use the same clib for the backup, there will be no additional
    overhead.


    Marcel

+ Reply to Thread