On 12/20/06 09:15, Arone Silimantia wrote:
> Eric,
>
> Thanks for your comments and help - your posts on this
> list are much appreciated. Comments in line below:
>
>
> --- Eric Anderson wrote:
>
>> With 9TB without
>> any journaling, you might run into problems if you
>> crash and need to
>> fsck - the number of files you could have on the
>> file system could well
>> require more memory/time than you have available.
>
>
> Hmmm...the time required is more dependent on inodes
> than on size of data / size of files, right ?
>
> My 9 TB dataset uses about 36 million inodes.
>
> Any comments on that number ? Large ? Pedestrian ?
> Typical ?

Sounds like you have a lot of larger files (~250k per file on average
possibly), which helps the fsck times. 36 million inodes should be
fsck'able with enough memory (maybe ~3gb ish? That's a wild guess). I
have two 10Tb file systems, one has 180million inodes. I don't attempt
to fsck it, because it would take a very long time, and I might possibly
run out of memory (I have 8Gb of memory). I use GJOURNAL on these, and
am very happy about it (thanks Pawel!).

df snippet:
Filesystem 1K-blocks Used Avail Capacity
iused ifree %iused Mounted on
/dev/label/vol10-data.journal 9925732858 8780187028 351487202 96%
180847683 1102147515 14% /vol10
/dev/label/vol11-data.journal 9925732858 2598987846 6532686384 28%
49422705 1233572493 4% /vol11

I also have several 2Tb partitions (set up prior to gjournal being
available), that are *FULL*, and each have about 25-45million inodes on
them. Those fsck in about 4-7hours each, using between 1Gb and 3Gb of
memory to do so.

>
> I am hoping that that could be fsck'd (modern hitachi
> SATA drives, raid-6, 3ware) in 48 hours ... or am I
> way off ?

Depends on the drives really, and maybe caching. The geom_cache module
(still beta probably, and not in the src tree currently I don't believe)
is said to improve fsck times. 48 hours is a long time, and I *think*
it should complete within that time frame, but you'd really have to test
it to be sure.

>>> But I do absolutely need to run quotas (both user
>> and
>>> group) on this 9 GB array. I also need to
>> successfully
>>> use all quota admin tools (repquota, edquota,
>> quota,
>>> etc.)
>>>
>>> Can I get an assurance that this is totally safe,
>>> sane, and fit to run in a mission critical, data
>>> critical environment ? Anyone doing it currently
>> ?
>>> Any comments or warnings of _any kind_ much
>> appreciated.
>>
>> I don't think anyone will say 'I promise it will
>> work' of course, but I
>> would start by using the latest 6-STABLE source
>> since there have been
>> quite a number of updates to file system related
>> code since 6.1.
>
>
> Ok, but all of the CLI tools (edquota, repquota,
> quota, quotacheck, quotaon) are all known-good for
> "bigdisk" ?
>
> And there is no known "quotas just don't work with
> bigdisk" problems ?
>
> I was hoping someone out there was running quotas with
> 6.1-RELEASE on a >2TB filesystem and could report favorably...

I'm not certain. There were some bugs in quotas, that recently were
fixed (Kris Kennaway I think reported them and saw the fixes into the
tree), and prior to that I saw those consistently, so I stopped using
them. I haven't tried since the fixes have been in place, and the fixes
(if I recall correctly) had to do with background fsck (softupdates
maybe) and not the size of the disk.

You could try this in a mock-up environment, by creating a sparse file
and use it with mdconfig, then newfs it, enabling quotas, mount, and
then use a script to create massive amounts (36million-ish) files of
about 200k-ish in size, with random users, in a similar fashion as your
real data, and see if all goes well. You can also try your fsck that
way too.

Eric



--
------------------------------------------------------------------------
Eric Anderson Sr. Systems Administrator Centaur Technology
An undefined problem has an infinite number of solutions.
------------------------------------------------------------------------
_______________________________________________
freebsd-fs@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"