Incorrect file size? - FreeBSD

This is a discussion on Incorrect file size? - FreeBSD ; -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello All, There is something strange on my mail server running 6.2-STABLE: # ls -l total 1239702 - -rw------- 1 vscan vscan 4398199488512 Jun 20 15:18 auto-whitelist - -rw------- 1 vscan vscan 22 Jun ...

+ Reply to Thread
Results 1 to 12 of 12

Thread: Incorrect file size?

  1. Incorrect file size?

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Hello All,

    There is something strange on my mail server running 6.2-STABLE:

    # ls -l
    total 1239702
    - -rw------- 1 vscan vscan 4398199488512 Jun 20 15:18 auto-whitelist
    - -rw------- 1 vscan vscan 22 Jun 20 15:18 bayes.lock
    - -rw------- 1 vscan vscan 102168 Jun 20 15:18 bayes_journal
    - -rw------- 1 vscan vscan 1099639861248 Jun 20 15:18 bayes_seen
    - -rw------- 1 vscan vscan 21057536 Jun 20 15:18 bayes_toks

    but:
    # df -h
    Filesystem Size Used Avail Capacity Mounted on
    /dev/ar0s1a 19G 3.9G 14G 22% /
    ....

    # du
    1239758 .

    # mount
    /dev/ar0s1a on / (ufs, local)
    ....

    When I try to delete some entries from auto-whitelist:
    Out of memory during request for 52 bytes, total sbrk() is 536223744 bytes!
    Out of memory during request for 56 bytes, total sbrk() is 536223744 bytes!

    Any ideas?
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.7 (FreeBSD)

    iD8DBQFIW6PRxJBWvpalMpkRAprHAKCZ93VqWsH28i0wa4bc+Z mXNYXF4gCdELF8
    EC/P28NCFIiws1yf+rci/ck=
    =O5TZ
    -----END PGP SIGNATURE-----
    _______________________________________________
    freebsd-stable@freebsd.org mailing list
    http://lists.freebsd.org/mailman/lis...freebsd-stable
    To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"


  2. Re: Incorrect file size?

    Hi,

    On Fri, Jun 20, 2008 at 03:34:26PM +0300, Krassimir Slavchev wrote:
    > # ls -l
    > total 1239702
    > - -rw------- 1 vscan vscan 4398199488512 Jun 20 15:18 auto-whitelist
    > - -rw------- 1 vscan vscan 22 Jun 20 15:18 bayes.lock
    > - -rw------- 1 vscan vscan 102168 Jun 20 15:18 bayes_journal
    > - -rw------- 1 vscan vscan 1099639861248 Jun 20 15:18 bayes_seen
    > - -rw------- 1 vscan vscan 21057536 Jun 20 15:18 bayes_toks
    >
    > but:
    > # df -h
    > Filesystem Size Used Avail Capacity Mounted on
    > /dev/ar0s1a 19G 3.9G 14G 22% /
    > ...
    >
    > # du
    > 1239758 .
    >
    > # mount
    > /dev/ar0s1a on / (ufs, local)
    > ...


    This is most likely due to the use of 'sparse files': sequential file
    data that consists of only zeros doesn't need have actual storage
    associated to it. This is quite normal in UNIX environments, and quite
    harmless.

    --
    Rink P.W. Springer - http://rink.nu
    "Anyway boys, this is America. Just because you get more votes doesn't
    mean you win." - Fox Mulder
    _______________________________________________
    freebsd-stable@freebsd.org mailing list
    http://lists.freebsd.org/mailman/lis...freebsd-stable
    To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"


  3. Re: Incorrect file size?

    Rink Springer wrote:
    > Hi,
    >
    > On Fri, Jun 20, 2008 at 03:34:26PM +0300, Krassimir Slavchev wrote:
    >> # ls -l
    >> total 1239702
    >> - -rw------- 1 vscan vscan 4398199488512 Jun 20 15:18 auto-whitelist
    >> - -rw------- 1 vscan vscan 22 Jun 20 15:18 bayes.lock
    >> - -rw------- 1 vscan vscan 102168 Jun 20 15:18 bayes_journal
    >> - -rw------- 1 vscan vscan 1099639861248 Jun 20 15:18 bayes_seen
    >> - -rw------- 1 vscan vscan 21057536 Jun 20 15:18 bayes_toks
    >>
    >> but:
    >> # df -h
    >> Filesystem Size Used Avail Capacity Mounted on
    >> /dev/ar0s1a 19G 3.9G 14G 22% /
    >> ...
    >>
    >> # du
    >> 1239758 .
    >>
    >> # mount
    >> /dev/ar0s1a on / (ufs, local)
    >> ...

    >
    > This is most likely due to the use of 'sparse files': sequential file
    > data that consists of only zeros doesn't need have actual storage
    > associated to it. This is quite normal in UNIX environments, and quite
    > harmless.


    Except that the file in question should be, judging by the filename, a
    simple text file. I don't really see how a whitelist could grow to such
    monstrous sizes Most likely it's a file system corruption - fsck
    should be the first thing to try.


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFIW7aOldnAQVacBcgRAouWAKCLxfKAx9767zfE+lIgFn 6rFZ4cawCeLstv
    Bkqut78WbRLWkrnMbUUq394=
    =lEs9
    -----END PGP SIGNATURE-----


  4. Re: Incorrect file size?

    On Fri, Jun 20, 2008 at 03:54:22PM +0200, Ivan Voras wrote:
    > Except that the file in question should be, judging by the filename, a
    > simple text file. I don't really see how a whitelist could grow to such
    > monstrous sizes Most likely it's a file system corruption - fsck
    > should be the first thing to try.


    The 'vscan' user leads me assume this is SpamAssassin - I've seen this
    behaviour at work, where our scripts were trying to backup a 1TB file
    (which actually was ~vscan/.spamassassin/auto-whitelist). The result was
    that the backup script died due to lack of disk space on the backup
    server (as we don't use compression).

    When I was investigating why the file could be so large it, it turned
    out the file was only a few hunderd 'real' MB's, so that is why I assume
    this person is having the same issue as we do. The file is a Berkeley DB
    file, by the way, so there's nothing textfile about it ;-)

    But still, when in doubt, fsck(8) is sure to aid you.

    --
    Rink P.W. Springer - http://rink.nu
    "Anyway boys, this is America. Just because you get more votes doesn't
    mean you win." - Fox Mulder
    _______________________________________________
    freebsd-stable@freebsd.org mailing list
    http://lists.freebsd.org/mailman/lis...freebsd-stable
    To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"


  5. Re: Incorrect file size?

    Rink Springer wrote:
    > On Fri, Jun 20, 2008 at 03:54:22PM +0200, Ivan Voras wrote:
    >> Except that the file in question should be, judging by the filename, a
    >> simple text file. I don't really see how a whitelist could grow to such
    >> monstrous sizes Most likely it's a file system corruption - fsck
    >> should be the first thing to try.

    >
    > The 'vscan' user leads me assume this is SpamAssassin - I've seen this
    > behaviour at work, where our scripts were trying to backup a 1TB file
    > (which actually was ~vscan/.spamassassin/auto-whitelist). The result was
    > that the backup script died due to lack of disk space on the backup
    > server (as we don't use compression).
    >
    > When I was investigating why the file could be so large it, it turned
    > out the file was only a few hunderd 'real' MB's, so that is why I assume
    > this person is having the same issue as we do. The file is a Berkeley DB
    > file, by the way, so there's nothing textfile about it ;-)


    I learn something every day
    Didn't know BDB was smart enough to create sparse files.


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFIW73BldnAQVacBcgRAueCAKDvHyIIKXlCeubk7H4Ddy hA76j9SACeLGUc
    BdGXkCi8uujS2fHqsRUtajg=
    =y1PG
    -----END PGP SIGNATURE-----


  6. Re: Incorrect file size?

    Ivan Voras wrote:
    > Rink Springer wrote:
    > > The 'vscan' user leads me assume this is SpamAssassin - I've seen this
    > > behaviour at work, where our scripts were trying to backup a 1TB file
    > > (which actually was ~vscan/.spamassassin/auto-whitelist). The result was
    > > that the backup script died due to lack of disk space on the backup
    > > server (as we don't use compression).
    > >
    > > When I was investigating why the file could be so large it, it turned
    > > out the file was only a few hunderd 'real' MB's, so that is why I assume
    > > this person is having the same issue as we do. The file is a Berkeley DB
    > > file, by the way, so there's nothing textfile about it ;-)

    >
    > I learn something every day
    > Didn't know BDB was smart enough to create sparse files.


    BTW, you can use "ls -ls" to display the number of physical
    blocks allocated to the file, so you can easily see whether
    a file is sparse or not:

    $ dd if=/dev/zero of=foo1 bs=1m count=1
    $ truncate -s 1m foo2
    $ ls -ls foo1 foo2
    1040 -rw------- 1 olli olli 1048576 Jun 20 22:43 foo1
    32 -rw------- 1 olli olli 1048576 Jun 20 22:43 foo2

    As you can see, the file size is the same, but the block
    counts are different (I have BLOCKSIZE=K in my environment,
    so the blocks are displayed in 1KB units).

    I've written a small script that can be used to detect
    sparse files (it even displays the "sparseness" percentage):

    http://www.secnetix.de/olli/scripts/sparsecheck

    Best regards
    Oliver

    PS: Of course it is still possible that a file system is
    corrupt and needs fsck, no matter whether those files are
    sparse or not.

    --
    Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
    Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
    secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
    chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

    FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd

    "C++ is over-complicated nonsense. And Bjorn Shoestrap's book
    a danger to public health. I tried reading it once, I was in
    recovery for months."
    -- Cliff Sarginson
    _______________________________________________
    freebsd-stable@freebsd.org mailing list
    http://lists.freebsd.org/mailman/lis...freebsd-stable
    To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"


  7. Re: Incorrect file size?

    On Fri, Jun 20, 2008 at 04:25:05PM +0200, Ivan Voras wrote:

    > > When I was investigating why the file could be so large it, it turned
    > > out the file was only a few hunderd 'real' MB's, so that is why I assume
    > > this person is having the same issue as we do. The file is a Berkeley DB
    > > file, by the way, so there's nothing textfile about it ;-)

    > I learn something every day
    > Didn't know BDB was smart enough to create sparse files.


    Even base system uses such files :-)

    $ ls -lsk /etc/pwd.db
    140 -rw-r--r-- 1 root wheel 159744 21 ÉÀÎ 12:26 /etc/pwd.db
    $ file /etc/pwd.db
    /etc/pwd.db: Berkeley DB 1.85 (Hash, version 2, native byte-order)

    File occupies 140K = 143360 bytes and its length is 159744 bytes,
    so it has to be sparse. FreeBSD 4.11-STABLE.

    Eugene
    _______________________________________________
    freebsd-stable@freebsd.org mailing list
    http://lists.freebsd.org/mailman/lis...freebsd-stable
    To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"


  8. Re: Incorrect file size?

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Oliver Fromme wrote:
    > Ivan Voras wrote:
    > > Rink Springer wrote:
    > > > The 'vscan' user leads me assume this is SpamAssassin - I've seen this
    > > > behaviour at work, where our scripts were trying to backup a 1TB file
    > > > (which actually was ~vscan/.spamassassin/auto-whitelist). The result was
    > > > that the backup script died due to lack of disk space on the backup
    > > > server (as we don't use compression).
    > > >
    > > > When I was investigating why the file could be so large it, it turned
    > > > out the file was only a few hunderd 'real' MB's, so that is why I assume
    > > > this person is having the same issue as we do. The file is a Berkeley DB
    > > > file, by the way, so there's nothing textfile about it ;-)

    > >
    > > I learn something every day
    > > Didn't know BDB was smart enough to create sparse files.

    >
    > BTW, you can use "ls -ls" to display the number of physical
    > blocks allocated to the file, so you can easily see whether
    > a file is sparse or not:
    >
    > $ dd if=/dev/zero of=foo1 bs=1m count=1
    > $ truncate -s 1m foo2
    > $ ls -ls foo1 foo2
    > 1040 -rw------- 1 olli olli 1048576 Jun 20 22:43 foo1
    > 32 -rw------- 1 olli olli 1048576 Jun 20 22:43 foo2


    # ls -lsk
    total 1247288
    664064 -rw------- 1 vscan vscan 4398199488512 Jun 23 09:39
    auto-whitelist
    88 -rw------- 1 vscan vscan 89976 Jun 23 09:39 bayes_journal
    566704 -rw------- 1 vscan vscan 1099639861248 Jun 23 09:39 bayes_seen
    16432 -rw------- 1 vscan vscan 21454848 Jun 23 09:39 bayes_toks

    >
    > As you can see, the file size is the same, but the block
    > counts are different (I have BLOCKSIZE=K in my environment,
    > so the blocks are displayed in 1KB units).
    >
    > I've written a small script that can be used to detect
    > sparse files (it even displays the "sparseness" percentage):
    >
    > http://www.secnetix.de/olli/scripts/sparsecheck
    >
    > Best regards
    > Oliver
    >
    > PS: Of course it is still possible that a file system is
    > corrupt and needs fsck, no matter whether those files are
    > sparse or not.
    >



    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.7 (FreeBSD)

    iD8DBQFIX0W+xJBWvpalMpkRAqrdAJ47eLQ+WMp6zBrme5gNyC SvzBtdUwCffYwT
    +37ul1gPqmk7rVKXRrha7fU=
    =uSGe
    -----END PGP SIGNATURE-----
    _______________________________________________
    freebsd-stable@freebsd.org mailing list
    http://lists.freebsd.org/mailman/lis...freebsd-stable
    To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"


  9. Re: Incorrect file size?

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Hi,

    Ivan Voras wrote:
    > Rink Springer wrote:
    >> On Fri, Jun 20, 2008 at 03:54:22PM +0200, Ivan Voras wrote:
    >>> Except that the file in question should be, judging by the filename, a
    >>> simple text file. I don't really see how a whitelist could grow to such
    >>> monstrous sizes Most likely it's a file system corruption - fsck
    >>> should be the first thing to try.

    >> The 'vscan' user leads me assume this is SpamAssassin - I've seen this
    >> behaviour at work, where our scripts were trying to backup a 1TB file
    >> (which actually was ~vscan/.spamassassin/auto-whitelist). The result was
    >> that the backup script died due to lack of disk space on the backup
    >> server (as we don't use compression).


    Yes, it is SpamAssassin and I have the same problem with the backups.
    The Amanda complains with:
    xxxxxxxxxxx / lev 1 FAILED [dump larger than available tape space,
    307256178 KB, skipping incremental]
    yyyyyyyyyyyy / lev 4 FAILED [dump larger than tape, 354005617 KB,
    skipping incremental]

    I don't think this is a file system corruption because there are no
    reasons for this. Also I have seen this on different machines.

    Any ideas how to fix this?

    >>
    >> When I was investigating why the file could be so large it, it turned
    >> out the file was only a few hunderd 'real' MB's, so that is why I assume
    >> this person is having the same issue as we do. The file is a Berkeley DB
    >> file, by the way, so there's nothing textfile about it ;-)

    >
    > I learn something every day
    > Didn't know BDB was smart enough to create sparse files.
    >




    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.7 (FreeBSD)

    iD8DBQFIX0kwxJBWvpalMpkRAjkaAKCbF7sylsDyI2umCAoneq BqAYCNqwCgnQjA
    9FkgJiVhqNI2NGCAlWpqfbg=
    =PQZ1
    -----END PGP SIGNATURE-----
    _______________________________________________
    freebsd-stable@freebsd.org mailing list
    http://lists.freebsd.org/mailman/lis...freebsd-stable
    To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"


  10. Re: Incorrect file size?

    On Mon, Jun 23, 2008 at 09:56:48AM +0300, Krassimir Slavchev wrote:

    > I don't think this is a file system corruption because there are no
    > reasons for this. Also I have seen this on different machines.
    >
    > Any ideas how to fix this?


    Either use backup software that is aware of sparse files
    or eliminate sofrware that uses such files :-)

    Eugene
    _______________________________________________
    freebsd-stable@freebsd.org mailing list
    http://lists.freebsd.org/mailman/lis...freebsd-stable
    To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"


  11. Re: Incorrect file size?

    On Mon, Jun 23, 2008 at 09:56:48AM +0300, Krassimir Slavchev wrote:
    > Ivan Voras wrote:
    > > Rink Springer wrote:
    > >> On Fri, Jun 20, 2008 at 03:54:22PM +0200, Ivan Voras wrote:
    > >>> Except that the file in question should be, judging by the filename, a
    > >>> simple text file. I don't really see how a whitelist could grow to such
    > >>> monstrous sizes Most likely it's a file system corruption - fsck
    > >>> should be the first thing to try.
    > >> The 'vscan' user leads me assume this is SpamAssassin - I've seen this
    > >> behaviour at work, where our scripts were trying to backup a 1TB file
    > >> (which actually was ~vscan/.spamassassin/auto-whitelist). The result was
    > >> that the backup script died due to lack of disk space on the backup
    > >> server (as we don't use compression).


    I've used SpamAssassin for a few years now, and I've never seen this
    happen (including during migration from RELENG_6 to RELENG_7, and
    between SpamAssassin versions (from 3.0 to 3.2.x). I cannot even begin
    to imagine how that file reached such a size, which is why I believe the
    issue may be filesystem corruption.

    I will take a moment to point out something I do whenever SpamAssassin
    gets upgraded, though: I always delete bayes_* and auto-whitelist files
    from .spamassassin, on every account. I do this because there have been
    cases in the past where the data format has changed in the DB, and SA
    hasn't done a good job of seamlessly migrating them.

    > Yes, it is SpamAssassin and I have the same problem with the backups.
    > The Amanda complains with:
    > xxxxxxxxxxx / lev 1 FAILED [dump larger than available tape space,
    > 307256178 KB, skipping incremental]
    > yyyyyyyyyyyy / lev 4 FAILED [dump larger than tape, 354005617 KB,
    > skipping incremental]


    There isn't any way to solve this problem. The file, as far as dump is
    concerned, is indeed 3TB.

    > I don't think this is a file system corruption because there are no
    > reasons for this. Also I have seen this on different machines.


    I can point you to all of the threads in the past year where users have
    had mysterious problems with their filesystems, which have been fixed by
    booting single-user and fsck'ing. That may not be the problem here,
    but it doesn't hurt for you to do it "just in case" now does it?

    If it happens on multiple machines, I've only two recommendations:

    1) Try what I said above (delete the bayes_* and auto-whitelist files).
    If SpamAssassin recreates them as 3TB, then there is indeed a problem
    with your system(s), because mine do not do that.

    2) Consider migrating to dspam instead, which supposedly has a much
    greater success rate of blocking spam.

    --
    | Jeremy Chadwick jdc at parodius.com |
    | Parodius Networking http://www.parodius.com/ |
    | UNIX Systems Administrator Mountain View, CA, USA |
    | Making life hard for others since 1977. PGP: 4BD6C0CB |

    _______________________________________________
    freebsd-stable@freebsd.org mailing list
    http://lists.freebsd.org/mailman/lis...freebsd-stable
    To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"


  12. Re: Incorrect file size?

    On Mon, Jun 23, 2008 at 12:19:28AM -0700, Jeremy Chadwick wrote:
    > I've used SpamAssassin for a few years now, and I've never seen this
    > happen (including during migration from RELENG_6 to RELENG_7, and
    > between SpamAssassin versions (from 3.0 to 3.2.x). I cannot even begin
    > to imagine how that file reached such a size, which is why I believe the
    > issue may be filesystem corruption.


    I've seen spamassassin creating large sparse file regurally and do
    other strang things like get the database into a state where it
    spins if it trys to clean it out. That's even starting with a clean
    database.

    > There isn't any way to solve this problem. The file, as far as dump is
    > concerned, is indeed 3TB.


    Dump does understand sparse files, but doesn't store them in the
    most efficient way possible. Dumping a filesystem with a 1TB file
    seems to use about 2GB.

    David.

    # truncate -s 1T /var/tmp/bigfile
    # dump 0Lf - /var | wc -c
    DUMP: Date of this level 0 dump: Mon Jun 23 09:50:50 2008
    DUMP: Date of last level 0 dump: the epoch
    DUMP: Dumping snapshot of /dev/ad4s3d (/var) to standard output
    DUMP: mapping (Pass I) [regular files]
    DUMP: mapping (Pass II) [directories]
    DUMP: estimated 2212849 tape blocks.
    DUMP: dumping (Pass III) [directories]
    DUMP: dumping (Pass IV) [regular files]
    DUMP: DUMP: 2212776 tape blocks
    DUMP: finished in 59 seconds, throughput 37504 KBytes/sec
    DUMP: DUMP IS DONE
    2265876480

    _______________________________________________
    freebsd-stable@freebsd.org mailing list
    http://lists.freebsd.org/mailman/lis...freebsd-stable
    To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"


+ Reply to Thread