[olug] Slow read speeds-SuSE

[Hurley Rod]

Craig,
I jumped into the middle of this thread, so this might be from left field BUT....

Aren't you doing iSCSI there?  It may be possible that you are experiencing packet fragmentation. When it builds the SCSI frame and adds the iSCI header, if the SCSI frame exceeds the IP frame size it will have to create a 2nd packet to send all the information.  This can pop up if you are not using JUMBO frames (and every single iSCSI network device and nic must support and be enabled for JUMBO frames, otherwise it will not support them).  When you are sending full frames without JUMBO, it will result in 2 packets instead of one, and of course double the I/O.  Just a thought.  

-----Original Message-----
From: olug-bounces at olug.org [mailto:olug-bounces at olug.org] On Behalf Of Craig Wolf
Sent: Thursday, April 29, 2010 9:43 AM
To: Omaha Linux User Group
Subject: Re: [olug] Slow read speeds-SuSE


Interesting...I copied the 158MB from HR_DATA to MAIN from the command line and it was instant (2 seconds).  So, you would think that it is NOT an IO problem, yes?  Makes me think that it is time to do a packet capture to see what the network is doing during a backup....
 


Craig Wolf
Linux Server Support
Backups Administrator
Desktop/Network Specialist
Desk: (402)715-6283
Cell: (402)510-0301



>>> Christopher Cashell <topher-olug at zyp.org> 4/28/2010 5:27 PM >>>
On Wed, Apr 28, 2010 at 4:16 PM, Craig Wolf <CJWolf at mpsomaha.org> wrote:
> Ok, my backups from one of my linux servers takes 5 days to finish...my window is 36 hours, not working right.  Throughput of about 1.5GB/hour.
> So in my testing, copying a 158MB directory to the server takes about 1 Minute 5 seconds.  Copying same structure off the box takes almost 7 minutes.  Network switches show no issues/error.  iostat shows slow to no throughput at all on read.
>
> SO, the million dollar question is, what should I look at for fixing this??  Do I use hdparm?

First thing, divide and conquer.  You need to isolate the problem.
Based on what you've said, the two likely culprits are slow disk or
slow network.

Try copying files from one partition to another partition (preferably
involving the partition where the backups are being read/written
to/from).  Check your copy speed.  Is it fast?  Is it slow?  If it's
fast, then it's not a disk issue.  If it's slow, we need to further
investigate the disk.

If it is fast, then it's probably a network issue.  Check the network
settings.  What kind of switch is this?  GigE?  Is the switch port and
server port configured the same?  What does ethtool or mii-tool tell
you about the interfaces?  If one side (server or switch) is set to
autonegotiate and the other side isn't, you'll often end up at 10BaseT
and half duplex.  If your network settings are all clean and good,
then test the network performance.  Use iperf or even just dd
/dev/zero into netcat to a remote box that listens with netcat and
redirects to /dev/null.

Hopefully that will give you a better idea of where to push forward
with further investigation.

> Craig Wolf

--
Christopher
_______________________________________________
OLUG mailing list
OLUG at olug.org
https://lists.olug.org/mailman/listinfo/olug
_______________________________________________
OLUG mailing list
OLUG at olug.org
https://lists.olug.org/mailman/listinfo/olug