IA-64 Linux Question on ProPack 3.0 - SGI
This is a discussion on IA-64 Linux Question on ProPack 3.0 - SGI ; Hello all,
we have two IRIX boxes (O2k and O3800) running 6.5.24 + latest patches, an IBM
p690 running AIX 5.2 and an Altix 3700 running PP3.0 + latest patches.
Our objective here is to share file space over NFS ...
-
NFS on IA-64 Linux and other UNIX systems
Hello all,
we have two IRIX boxes (O2k and O3800) running 6.5.24 + latest patches, an IBM
p690 running AIX 5.2 and an Altix 3700 running PP3.0 + latest patches.
Our objective here is to share file space over NFS accross systems. So far it
appears that IRIX can BOTH serve and mount NFS from each one of the systems
(including some RH 9.0 and Solaris boxes) but AIX has problems using NFS
exported file systems from the Altix and the other RH Linux boxes.
Is there any particular option that needs to be enabled in order for NFS to
work across AIX and Linux?
Thanks
Michael
On Tue, 1 Jun 2004, Alexis Cousein wrote:
a' Date: Tue, 01 Jun 2004 16:22:21 +0200
a' From: Alexis Cousein
a' To: Michael E. Thomadakis
a' Newsgroups: comp.sys.sgi.admin, comp.os.linux.setup
a' Subject: Re: NFS on IA-64 Linux
a'
a' Michael E. Thomadakis wrote:
a'
a' > Hello all.
a' >
a' > We are maintaining an SGI Altix 3700 (128 PE, 10TB RAID storage) currently
a' > running PP2.4.
a' >
a' > Has anyone had experience with using this system as an NFS server?
a'
a' It works well.
a'
a' Provided the clients use read and write sizes that are
a' a multiple of the base page size -- which, on this machine, is *16KB* -
a' the default 8KB xfer sizes are a no-no (and an even bigger no-no for
a' the NFS client, which means you do have to patch up many IA32 Linux
a' servers quite heavily if they have to act as an NFS server for an
a' Altix -- many with a 2.4 kernel do not really support sizes over 8KB,
a' though Propack 2.4 does).
a'
a' The same advice holds for any other Linux server using a 2.4 kernel -
a' but of course, 8KB is a multiple of a 4KB base page size on most
a' IA32 Linux servers...
a'
a' As a consequence, NFS v2 is also a big no-no if you want performance,
a' given its insistence on limiting xfer size to 8KB.
a'
a'
a' > There is
a' > not much discussion for NFS in PP2.4. Can TCP be used as the transport for
a' > NFS?
a'
a' Not without patch 10054. Even with that patch, the 2.4 client code
a' is noticeably faster over UDP - unless, of course, you have a dodgy
a' network that can't get a 16KB transfer size over reliably, but
a' manages to get 1500 byte MTUs over most of the time.
a'
a' Is there any problem with large files/file systems?
a'
a' Not unless you're mounting with the "soft" option. There are
a' a few spots in the (generic) 2.4 client code that can trigger
a' bogus timeouts propagated to the application as errors
a' if you use this option.
a'
a' Don't export with the "async" option (this is generic advice) if
a' all your clients are using NFS v3.
a'
a' There's a performance divot for non root copies of files with the group
a' execute permission bit set if you don't have patch 10065.
a'
a' > How is th
a' > throughput?
a'
a' Throughput is OK for one interface. Scalability for many GigE interfaces
a' is poor if you're used to IRIX, and less good than on 2.6 kernels
a' (by a huge margin), and even in 2.6 there's quite some work cut out to
a' achieve IRIX-level scalability.
a'
a' --
a' Alexis Cousein Senior Systems Engineer
a' alexis@sgi.com SGI/Silicon Graphics Brussels
a'
a' If I have seen further, it is by standing on reference manuals.
a'
a'
-
Re: NFS on IA-64 Linux and other UNIX systems
Michael E. Thomadakis wrote:
> Hello all,
>
> we have two IRIX boxes (O2k and O3800) running 6.5.24 + latest patches, an IBM
> p690 running AIX 5.2 and an Altix 3700 running PP3.0 + latest patches.
>
> Our objective here is to share file space over NFS accross systems. So far it
> appears that IRIX can BOTH serve and mount NFS from each one of the systems
> (including some RH 9.0 and Solaris boxes) but AIX has problems using NFS
> exported file systems from the Altix and the other RH Linux boxes.
>
> Is there any particular option that needs to be enabled in order for NFS to
> work across AIX and Linux?
>
> Thanks
> Michael
>
> On Tue, 1 Jun 2004, Alexis Cousein wrote:
>
> a' Date: Tue, 01 Jun 2004 16:22:21 +0200
> a' From: Alexis Cousein
> a' To: Michael E. Thomadakis
> a' Newsgroups: comp.sys.sgi.admin, comp.os.linux.setup
> a' Subject: Re: NFS on IA-64 Linux
> a'
> a' Michael E. Thomadakis wrote:
> a'
> a' > Hello all.
> a' >
> a' > We are maintaining an SGI Altix 3700 (128 PE, 10TB RAID storage) currently
> a' > running PP2.4.
> a' >
> a' > Has anyone had experience with using this system as an NFS server?
> a'
> a' It works well.
> a'
> a' Provided the clients use read and write sizes that are
> a' a multiple of the base page size -- which, on this machine, is *16KB* -
> a' the default 8KB xfer sizes are a no-no (and an even bigger no-no for
> a' the NFS client, which means you do have to patch up many IA32 Linux
> a' servers quite heavily if they have to act as an NFS server for an
> a' Altix -- many with a 2.4 kernel do not really support sizes over 8KB,
> a' though Propack 2.4 does).
> a'
> a' The same advice holds for any other Linux server using a 2.4 kernel -
> a' but of course, 8KB is a multiple of a 4KB base page size on most
> a' IA32 Linux servers...
> a'
> a' As a consequence, NFS v2 is also a big no-no if you want performance,
> a' given its insistence on limiting xfer size to 8KB.
> a'
> a'
> a' > There is
> a' > not much discussion for NFS in PP2.4. Can TCP be used as the transport for
> a' > NFS?
> a'
> a' Not without patch 10054. Even with that patch, the 2.4 client code
> a' is noticeably faster over UDP - unless, of course, you have a dodgy
> a' network that can't get a 16KB transfer size over reliably, but
> a' manages to get 1500 byte MTUs over most of the time.
> a'
> a' Is there any problem with large files/file systems?
> a'
> a' Not unless you're mounting with the "soft" option. There are
> a' a few spots in the (generic) 2.4 client code that can trigger
> a' bogus timeouts propagated to the application as errors
> a' if you use this option.
> a'
> a' Don't export with the "async" option (this is generic advice) if
> a' all your clients are using NFS v3.
> a'
> a' There's a performance divot for non root copies of files with the group
> a' execute permission bit set if you don't have patch 10065.
> a'
> a' > How is th
> a' > throughput?
> a'
> a' Throughput is OK for one interface. Scalability for many GigE interfaces
> a' is poor if you're used to IRIX, and less good than on 2.6 kernels
> a' (by a huge margin), and even in 2.6 there's quite some work cut out to
> a' achieve IRIX-level scalability.
> a'
> a' --
> a' Alexis Cousein Senior Systems Engineer
> a' alexis@sgi.com SGI/Silicon Graphics Brussels
> a'
> a' If I have seen further, it is by standing on reference manuals.
> a'
> a'
Michael,
Without a description of the problem, it is difficult to help. By
default, some of the AIX daemons run on random ports. Perhaps the linux
firewall is blocking access. You can check this by disabling the
firewall on linux using "iptables -F".
-
Re: NFS on IA-64 Linux and other UNIX systems
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
NotDashEscaped: You need GnuPG to verify this message
In comp.os.linux.setup Michael E. Thomadakis suggested:
> Hello all,
> we have two IRIX boxes (O2k and O3800) running 6.5.24 + latest patches, an IBM
> p690 running AIX 5.2 and an Altix 3700 running PP3.0 + latest patches.
> Our objective here is to share file space over NFS accross systems. So far it
> appears that IRIX can BOTH serve and mount NFS from each one of the systems
> (including some RH 9.0 and Solaris boxes) but AIX has problems using NFS
> exported file systems from the Altix and the other RH Linux boxes.
> Is there any particular option that needs to be enabled in order for NFS to
> work across AIX and Linux?
Try the Linux NFS HOWTO (www.tldp.org) it contains a bunch of
hints for many other *nix to get things working probably.
Good luck
--
Michael Heiming (GPG-Key ID: 0xEDD27B94)
mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
iD8DBQFA7EAVAkPEju3Se5QRAugLAKCO6H60YQBPa3+esPjNth iUHlGHlACePDOZ
olzcqZshWGUSwGFHsW3Xvw0=
=S3D7
-----END PGP SIGNATURE-----
-
Re: NFS on IA-64 Linux and other UNIX systems
On Wed, 07 Jul 2004 00:03:51 -0500, Michael E. Thomadakis wrote:
> Is there any particular option that needs to be enabled in order for NFS
> to work across AIX and Linux?
The most common problem with NFS and AIX and Linux is the AIX (>=4.2.1)
NFS client's perverse (IMHO) default of using unpriveleged source ports.
Linux doesn't like this and refuses the mount request (older systems used
to put out a message about "psychoanalysis suggested" but I guess they
thought that looked a little unprofessional).
The solution is to use "nfso -o nfs_use_reserved_ports=1" on AIX. I put it
in /etc/rc.nfs as well as issuing it on the command line one off.
I don't know about Altix. It may have the same fussiness as Linux. It is
not a bad thing to be fussy about, as allowing client requests on
unpriveleged ports enables a non root program to mount the filesystem, and
gain root privelege on the server if it's exported for root. Most other
UNIX systems (e.g. Solaris) are not fussy, although they don't use
unpriveleged ports on client requests either. The only one I have
personally used which behaves like Linux in this respect is Siemens'
Reliant UNIX which you don't meet very often these days.
When mounting a filesystem on a Linux client from a UNIX server (certainly
from Solaris or AIX) it is as well to specify rsize=8192,wsize=8192 in the
mount options. It improves performance considerably. Otherwise I've not
seen a problem when Linux is the client.
Regards, Ian
-
Re: NFS on IA-64 Linux and other UNIX systems
On Wed, 7 Jul 2004, Mike O'Reilly wrote:
m% Date: Wed, 07 Jul 2004 09:48:56 -0700
m% From: Mike O'Reilly
m% Newsgroups: comp.sys.sgi.admin, comp.os.linux.setup, comp.unix.aix
m% Subject: Re: NFS on IA-64 Linux and other UNIX systems
m%
m% Michael E. Thomadakis wrote:
m% > Hello all,
m% >
[ ... ]
m%
m% Michael,
m%
m% Without a description of the problem, it is difficult to help. By
m% default, some of the AIX daemons run on random ports. Perhaps the linux
m% firewall is blocking access. You can check this by disabling the
m% firewall on linux using "iptables -F".
That was the problem: the AIX NFS client was not using ports < 1024 and th
Linux wouldn't oblige. Thanks
-Michael
m%
m%
-
Re: NFS on IA-64 Linux and other UNIX systems
On Wed, 7 Jul 2004, Michael Heiming wrote:
m' Date: Wed, 7 Jul 2004 18:25:26 -0000
m' From: Michael Heiming
m' Newsgroups: comp.sys.sgi.admin, comp.os.linux.setup, comp.unix.aix
m' Subject: Re: NFS on IA-64 Linux and other UNIX systems
m'
[ ... ]
m'
m' Try the Linux NFS HOWTO (www.tldp.org) it contains a bunch of
m' hints for many other *nix to get things working probably.
m'
m' Good luck
I had been visiting these places but to no avail
Thanks
-Michael
m'
m' --
m' Michael Heiming (GPG-Key ID: 0xEDD27B94)
m' mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
m' -----BEGIN PGP SIGNATURE-----
m' Version: GnuPG v1.2.1 (GNU/Linux)
m'
m' iD8DBQFA7EAVAkPEju3Se5QRAugLAKCO6H60YQBPa3+esPjNth iUHlGHlACePDOZ
m' olzcqZshWGUSwGFHsW3Xvw0=
m' =S3D7
m' -----END PGP SIGNATURE-----
m'
-
Re: NFS on IA-64 Linux and other UNIX systems
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
NotDashEscaped: You need GnuPG to verify this message
In comp.os.linux.setup Michael E. Thomadakis suggested:
> On Wed, 7 Jul 2004, Mike O'Reilly wrote:
[..]
> That was the problem: the AIX NFS client was not using ports < 1024 and th
> Linux wouldn't oblige. Thanks
Exactly this problems and how to enable is mentioned in the Linux
NFS Howto, I suggested...
--
Michael Heiming (GPG-Key ID: 0xEDD27B94)
mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
iD8DBQFA7QfJAkPEju3Se5QRApHoAJ0Z7zCN06B4hMTI/nqXJj7bd78d0gCfRFYk
juvAyO24WXG0Cv/tFM1IjGg=
=MwG7
-----END PGP SIGNATURE-----
-
Re: NFS on IA-64 Linux and other UNIX systems
Michael Heiming wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> NotDashEscaped: You need GnuPG to verify this message
>
> In comp.os.linux.setup Michael E. Thomadakis suggested:
>
>
>
>>On Wed, 7 Jul 2004, Mike O'Reilly wrote:
>
>
> [..]
>
>
>>That was the problem: the AIX NFS client was not using ports < 1024 and th
>>Linux wouldn't oblige. Thanks
>
>
> Exactly this problems and how to enable is mentioned in the Linux
> NFS Howto, I suggested...
>
http://www.tldp.org/HOWTO/NFS-HOWTO/interop.html#AIX to be specific.
-
Re: NFS on IA-64 Linux and other UNIX systems
On Thu, 8 Jul 2004, Jeff Hanson wrote:
j) Date: Thu, 08 Jul 2004 08:59:16 -0400
j) From: Jeff Hanson
j) Newsgroups: comp.sys.sgi.admin, comp.os.linux.setup, comp.unix.aix
[ ... ]
j) >
j) >>That was the problem: the AIX NFS client was not using ports < 1024 and th
j) >>Linux wouldn't oblige. Thanks
j) >
j) >
j) > Exactly this problems and how to enable is mentioned in the Linux
j) > NFS Howto, I suggested...
j) >
j) http://www.tldp.org/HOWTO/NFS-HOWTO/interop.html#AIX to be specific.
I had seen in the past the various NFS HOWTOs cached in different places
around the web and I saw this reference to AIX 4.3.x. The NFS on AIX 5.2 (that
we run) is a much more improved implementation and I thought that it wouldn't
apply to us.
Thanks though, as this reading refers to other OSs as well.
-MT
-
Re: NFS on IA-64 Linux and other UNIX systems
Michael E. Thomadakis wrote:
>
> On Wed, 7 Jul 2004, Michael Heiming wrote:
>
> m' Date: Wed, 7 Jul 2004 18:25:26 -0000
> m' From: Michael Heiming
> m' Newsgroups: comp.sys.sgi.admin, comp.os.linux.setup, comp.unix.aix
> m' Subject: Re: NFS on IA-64 Linux and other UNIX systems
> m'
> [ ... ]
> m'
> m' Try the Linux NFS HOWTO (www.tldp.org) it contains a bunch of
> m' hints for many other *nix to get things working probably.
> m'
> m' Good luck
>
> I had been visiting these places but to no avail
>
> Thanks
> -Michael
>
> m'
> m' --
> m' Michael Heiming (GPG-Key ID: 0xEDD27B94)
> m' mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
> m' -----BEGIN PGP SIGNATURE-----
> m' Version: GnuPG v1.2.1 (GNU/Linux)
> m'
> m' iD8DBQFA7EAVAkPEju3Se5QRAugLAKCO6H60YQBPa3+esPjNth iUHlGHlACePDOZ
> m' olzcqZshWGUSwGFHsW3Xvw0=
> m' =S3D7
> m' -----END PGP SIGNATURE-----
> m'
Michael,
Here is the solution to the problem I had run into:
FIX
You need to open up the Linux firewall to accept udp packets from the
host and port that mountd is running on.
By default, on an AIX V5.1 system mountd runs on a random port. To limit
the number of ports that need to be opened on the Linux system one can
add the following lines to /etc/services on the AIX system.
mountd 30000/udp
mountd 30000/tcp
Next accept udp packets originating from port 30000 on the AIX system. A
fairly restrictive rule that would work, accepting udp packets only from
port 30000 of the AIX NFS server, would be:
iptables -I INPUT 1 -s 9.47.9.61 -p udp --sport 30000 -j ACCEPT
-
Re: NFS on IA-64 Linux and other UNIX systems
On Thu, 8 Jul 2004, Mike O'Reilly wrote:
o' Date: Thu, 08 Jul 2004 16:23:56 -0700
o' From: Mike O'Reilly
o' Newsgroups: comp.sys.sgi.admin, comp.os.linux.setup, comp.unix.aix
o' Subject: Re: NFS on IA-64 Linux and other UNIX systems
o'
o' [ ... ]
o' Michael,
o'
o' Here is the solution to the problem I had run into:
o'
o' FIX
o'
o' You need to open up the Linux firewall to accept udp packets from the
o' host and port that mountd is running on.
o'
o' By default, on an AIX V5.1 system mountd runs on a random port. To limit
o' the number of ports that need to be opened on the Linux system one can
o' add the following lines to /etc/services on the AIX system.
o'
o' mountd 30000/udp
o' mountd 30000/tcp
o'
o' Next accept udp packets originating from port 30000 on the AIX system. A
o' fairly restrictive rule that would work, accepting udp packets only from
o' port 30000 of the AIX NFS server, would be:
o'
o' iptables -I INPUT 1 -s 9.47.9.61 -p udp --sport 30000 -j ACCEPT
Mike that's a good suggestion to keep in mind. We do run iptables on the Linux
(Altix and RH9.0) but I had to stop it for other reasons. After I reanable it,
I will make sure that the various NFS ports are not filtered.
Thanks!
-MT