[olug] Server Farms

[Nick Walter]

Here's a few generic tips if you are getting into a serious server farm
for the first time.

1.  Good facilities are critical.  Nothing like seeing 70 servers crash
at once because the datacenter power circuits weren't up to handling the
load.  Ditto for the datacenter Cooling, phone, local LAN, and net
uplinks.  Facilities can cost some serious $$$ so make sure someone is
considering the cost of facilities when bidding the project.

2.  Monitoring tools are important.  There are commercial ones or you
can roll your own with shell scripts, perl, etc. if the app is home
grown.  At a minimum, there should be some facility for immediate beeper
notification of an on-call tech for any critical problem.  

3.  Standardization is really really helpful.  If a datacenter has to
monitor 90 servers and in that group of 90 there are  82 different
hardware configs then nightmares will ensue.  If there are 2 different
models of server then things are better.

4.  Design for redundancy.  Perhaps use high availability clustering
techniques or perhaps just have spare servers sitting around ready to be
re-purposed to cover for downed nodes.  

5.  Take safety into account.  NEBS compliant servers and proper fire
extinguishing systems also add a lot $$$ cost.  Keep this in mind when
bidding.

6.  Backups are important.  This should be an obvious point, but backups
can become nightmarish when there 10 databases and 80 application
servers all needing full backups on a weekly basis.  Plan on spending
some $$$ for a robotic tape library and a good set of backup software.

7.  Consultants aren't a bad idea.  Normally I'm a believer in having
skills in-house and not being hostage to outside expertise, but I make
exceptions for projects like this.  When it is important to expand into
a new area quickly and without growing pains, paying an outside expert
to mentor internal personnel is a good way to go.

Nick Walter


On Wed, 2004-02-25 at 12:14, Shaughn wrote:
> Hey. My boss just walked up to me and asked if I can build him a 
> 1.4million dollar server farm..
> 
> i can't...
> 
> 
> the price might not be that high, but apparently is that extreme.
> 
> I'm just wondering if any of you have any experience with server farms 
> of any variant, what the best one would be / etc..
> 
> I've seen stuff about BSD, and LINUX.. and sparcs and such..
> 
> just don't know where to start..
> 
> Thanks
> 
> 
> 
> _______________________________________________
> OLUG mailing list
> OLUG at olug.org
> http://lists.olug.org/mailman/listinfo/olug
>