[olug] nfs2 or nfs3?

VincentR vincentr at cox.net
Sat Feb 23 03:18:23 UTC 2002 

BTW, I made a dirty little script for bonnie testing.  If anyone wants it, it's here.
It's meant to run the bonnie test on nfs mounts with different file sizes at a range of rsize/wsize mount options.
http://members.cox.net/drizzt/bonnie_tests

Help yourself to any of the other stuff out there as well (like the IFERR script which caught my problem below).
Have fun!

----- Original Message -----
From: "VincentR" <vincentr at cox.net>
To: <olug at bstc.net>
Sent: Friday, February 22, 2002 8:53 PM
Subject: Re: [olug] nfs2 or nfs3?


> Well, I figured out my problem.  As usual, the most confusing problems have simple answers.  It was a typo on
> modules.conf which was letting the interface run at half duplex while others on the switch were at full duplex.  The
> impact on speed was only really apparent when benchmarking it against other systems.
> Anyway, I learned quite a bit about optimizing nfs.
> Here are the kernel parameters (add to start case in nfs service script, restore to default in the stop case) which I
> found to provide some improvement when running dd, bonnie and iozone testing:
>         echo "40 0 0 0 60 300 60 0 0" > /proc/sys/vm/bdflush
>         echo "262144" > /proc/sys/net/core/rmem_default
>         echo "262144" > /proc/sys/net/core/rmem_max
>         echo "524288" > /proc/sys/net/ipv4/ipfrag_high_thresh
>         echo "393216" > /proc/sys/net/ipv4/ipfrag_low_thresh
>         echo "90" > /proc/sys/net/ipv4/ipfrag_time
>
> Here are the mount/export options which I found to be the fastest:
> mount -t nfs -o defaults,async,nfsvers=3,rw,noatime,rsize=16384,wsize=16384,soft,udp,lock fse2:/pub /mnt/point
>
> Here are the nfs export options I used:
> [root at fse2 root]# cat /etc/exports
> /pub    10.10.10.0/24(rw,no_root_squash,async,wdelay)
> /pub    10.210.10.0/24(ro,no_root_squash,sync,no_wdelay)
>
>
> Your mileage may vary.
>
> ----- Original Message -----
> From: "VincentR" <vincentr at cox.net>
> To: <olug at bstc.net>
> Sent: Wednesday, February 20, 2002 11:42 PM
> Subject: Re: [olug] nfs2 or nfs3?
>
>
> > Yeah, I've defiantly ruled out hardware as a problem and the network isn't an issue either.  All of these systems
are
> on
> > the same switch.  Just as a rudimentary test, I ran ping -n -f -c 500 -s 32765 (32K) to and from both systems.  The
> > packet loss was less than 3%.  I also tweaked the ipfrag_high/low and time values in /proc as well as setting
bdflush
> to
> > .6 seconds rather that the 3 sec default.  None of that was beneficial.
> >
> > I was planning to stay away from the new vm stuff in the 2.4.10 and higher kernels.  I had read many concerns about
> the
> > stability of both ext3 and nfs in those releases.  Some other reading said most things have been ironed out in
2.4.17,
> > but nothing specific regarding ext3 and nfs.
> >
> > It looks like I'll be patching and poking 2.4.17 and see how that goes tomorrow.  I'll let you know how it turns
out.
> > Has anyone else has experiences with the new vm kernels?  What does this rmap patch provide?
> >
> > ----- Original Message -----
> > From: "Phil Brutsche" <phil at giedi.obix.com>
> > To: <olug at bstc.net>
> > Sent: Wednesday, February 20, 2002 11:07 PM
> > Subject: Re: [olug] nfs2 or nfs3?
> >
> >
> > > On Wed, 2002-02-20 at 21:15, VincentR wrote:
> > > > It turns out that I am using nfs3.
> > >
> > > Ok
> > >
> > > > The new system I'm building is only half as fast as the old one when
> > > > writing data.  It's the same when reading.  The nfs configuration is
> > > > identical on both systems.
> > >
> > > Something *is* odd.  I'm consistently getting 8.6 megabytes/sec writing.
> > >
> > > I'm using 2.4.17 + the rmap-11c VM patch.  NFS server has a pair of 40GB
> > > IDE drive connected to a 3ware 6400 RAID controller, and is a 450Mhz PII
> > > w/ 768 MB RAM.
> > >
> > > > I've tried optimizing many things in proc.  I've even recompiled the
> > > > kernel with nfs built in and 32K max wsize in .../nfsd/const.h.  I've
> > > > tried every combination of things, different r/wsizes, mount options,
> > > > etc...
> > > > The new system should be faster in every aspect.  It has dual P3
> > > > 800Mhz, 1GB Reg./ECC SDRAM, SCSI 160.  The old system (fse) is just a
> > > > single P3 600, 512MB SDRAM and ide drives.
> > >
> > > Neither the memory, nor the disks, are a problem - your working set for
> > > your test can be (and probably is) cached entirely in RAM.  The problem
> > > lies elsewhere...
> > >
> > > What lies on the network between the two computers?  I've heard that UDP
> > > packet fragmentation can cause performance problems with NFS.
> > >
> > > > I've tested the new one with bonnie and the disks get 79MB throughput,
> > > > so that's not the bottleneck.  The network isn't the problem either,
> > > > The difference between the two are 2.2.14->2.4.9smp kernel and ext3.
> > >
> > > Ah - it's RedHat.
> > >
> > > Have you considered trying 2.4.17 with either the rmap or aa VM patches?
> > >
> > >
> > > Phil
> > >
> > >
> > > -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
> > >
> > > For help contact olug-help at bstc.net - run by ezmlm
> > > to unsubscribe, send mail to olug-unsubscribe at bstc.net
> > > or `mail olug-unsubscribe at bstc.net < /dev/null`
> > > (c)2001 OLUG http://www.olug.org
> > >
> > > -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
> > >
> >
> >
> > -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
> >
> > For help contact olug-help at bstc.net - run by ezmlm
> > to unsubscribe, send mail to olug-unsubscribe at bstc.net
> > or `mail olug-unsubscribe at bstc.net < /dev/null`
> > (c)2001 OLUG http://www.olug.org
> >
> > -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
> >
>
>
> -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
>
> For help contact olug-help at bstc.net - run by ezmlm
> to unsubscribe, send mail to olug-unsubscribe at bstc.net
> or `mail olug-unsubscribe at bstc.net < /dev/null`
> (c)2001 OLUG http://www.olug.org
>
> -_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
>


-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_

For help contact olug-help at bstc.net - run by ezmlm
to unsubscribe, send mail to olug-unsubscribe at bstc.net
or `mail olug-unsubscribe at bstc.net < /dev/null`
(c)2001 OLUG http://www.olug.org

-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_

More information about the OLUG mailing list