[olug] nfs2 or nfs3?

[VincentR]

Well, I figured out my problem.  As usual, the most confusing problems have simple answers.  It was a typo on
modules.conf which was letting the interface run at half duplex while others on the switch were at full duplex.  The
impact on speed was only really apparent when benchmarking it against other systems.
Anyway, I learned quite a bit about optimizing nfs.
Here are the kernel parameters (add to start case in nfs service script, restore to default in the stop case) which I
found to provide some improvement when running dd, bonnie and iozone testing:
        echo "40 0 0 0 60 300 60 0 0" > /proc/sys/vm/bdflush
        echo "262144" > /proc/sys/net/core/rmem_default
        echo "262144" > /proc/sys/net/core/rmem_max
        echo "524288" > /proc/sys/net/ipv4/ipfrag_high_thresh
        echo "393216" > /proc/sys/net/ipv4/ipfrag_low_thresh
        echo "90" > /proc/sys/net/ipv4/ipfrag_time

Here are the mount/export options which I found to be the fastest:
mount -t nfs -o defaults,async,nfsvers=3,rw,noatime,rsize=16384,wsize=16384,soft,udp,lock fse2:/pub /mnt/point

Here are the nfs export options I used:
[root at fse2 root]# cat /etc/exports
/pub    10.10.10.0/24(rw,no_root_squash,async,wdelay)
/pub    10.210.10.0/24(ro,no_root_squash,sync,no_wdelay)


Your mileage may vary.

----- Original Message -----
From: "VincentR" <vincentr at cox.net>
To: <olug at bstc.net>
Sent: Wednesday, February 20, 2002 11:42 PM
Subject: Re: [olug] nfs2 or nfs3?


> Yeah, I've defiantly ruled out hardware as a problem and the network isn't an issue either.  All of these systems are
on
> the same switch.  Just as a rudimentary test, I ran ping -n -f -c 500 -s 32765 (32K) to and from both systems.  The
> packet loss was less than 3%.  I also tweaked the ipfrag_high/low and time values in /proc as well as setting bdflush
to
> .6 seconds rather that the 3 sec default.  None of that was beneficial.
>
> I was planning to stay away from the new vm stuff in the 2.4.10 and higher kernels.  I had read many concerns about
the
> stability of both ext3 and nfs in those releases.  Some other reading said most things have been ironed out in 2.4.17,
> but nothing specific regarding ext3 and nfs.
>
> It looks like I'll be patching and poking 2.4.17 and see how that goes tomorrow.  I'll let you know how it turns out.
> Has anyone else has experiences with the new vm kernels?  What does this rmap patch provide?
>
> ----- Original Message -----
> From: "Phil Brutsche" <phil at giedi.obix.com>
> To: <olug at bstc.net>
> Sent: Wednesday, February 20, 2002 11:07 PM
> Subject: Re: [olug] nfs2 or nfs3?
>
>
> > On Wed, 2002-02-20 at 21:15, VincentR wrote:
> > > It turns out that I am using nfs3.
> >
> > Ok
> >
> > > The new system I'm building is only half as fast as the old one when
> > > writing data.  It's the same when reading.  The nfs configuration is
> > > identical on both systems.
> >
> > Something *is* odd.  I'm consistently getting 8.6 megabytes/sec writing.
> >
> > I'm using 2.4.17 + the rmap-11c VM patch.  NFS server has a pair of 40GB
> > IDE drive connected to a 3ware 6400 RAID controller, and is a 450Mhz PII
> > w/ 768 MB RAM.
> >
> > > I've tried optimizing many things in proc.  I've even recompiled the
> > > kernel with nfs built in and 32K max wsize in .../nfsd/const.h.  I've
> > > tried every combination of things, different r/wsizes, mount options,
> > > etc...
> > > The new system should be faster in every aspect.  It has dual P3
> > > 800Mhz, 1GB Reg./ECC SDRAM, SCSI 160.  The old system (fse) is just a
> > > single P3 600, 512MB SDRAM and ide drives.
> >
> > Neither the memory, nor the disks, are a problem - your working set for
> > your test can be (and probably is) cached entirely in RAM.  The problem
> > lies elsewhere...
> >
> > What lies on the network between the two computers?  I've heard that UDP
> > packet fragmentation can cause performance problems with NFS.
> >
> > > I've tested the new one with bonnie and the disks get 79MB throughput,
> > > so that's not the bottleneck.  The network isn't the problem either,
> > > The difference between the two are 2.2.14->2.4.9smp kernel and ext3.
> >
> > Ah - it's RedHat.
> >
> > Have you considered trying 2.4.17 with either the rmap or aa VM patches?
> >
> >
> > Phil
> >
> >
> > -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
> >
> > For help contact olug-help at bstc.net - run by ezmlm
> > to unsubscribe, send mail to olug-unsubscribe at bstc.net
> > or `mail olug-unsubscribe at bstc.net < /dev/null`
> > (c)2001 OLUG http://www.olug.org
> >
> > -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
> >
>
>
> -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
>
> For help contact olug-help at bstc.net - run by ezmlm
> to unsubscribe, send mail to olug-unsubscribe at bstc.net
> or `mail olug-unsubscribe at bstc.net < /dev/null`
> (c)2001 OLUG http://www.olug.org
>
> -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
>


-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_

For help contact olug-help at bstc.net - run by ezmlm
to unsubscribe, send mail to olug-unsubscribe at bstc.net
or `mail olug-unsubscribe at bstc.net < /dev/null`
(c)2001 OLUG http://www.olug.org

-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_