[WEB4LIB] Web Server Fault Tolerance/Load Sharing

[sean dreilinger]

funny how web servers go from an experiment to a mission-critical
service requiring high-availability almost overnight :-) in the past i
checked into similar questions for clients of interactivate consulting
group, summary of what i learned is below.

would love to know what you end up doing!

--sean

mailto:sean at savvysearch.com                sean dreilinger, mlis
 http://www.savvysearch.com                http://durak.org/sean


Donna Schumann wrote:
> What do you need to set up a "backup" web server to automatically take
> over if the primary web server fails?

practical concerns to do this would include:

1. content - a system to keep content/configuration in sync across one
   or more servers.
2. service - a system to detect a fried server and redirect traffic to a
   working server.

1. content - to keep content consistent across two or more machines you
might use something simple like rsync (url below) or something fancier
to manage the entire web authoring process, like cvs (url below). rsync
runs on win32/unix and cvs runs on win32/unix/macintosh and then some.
both are free. if you have a monster budget you can also check out
storage area networks, e.g. a big fat RAID to hold your content and
allow several servers to access it at once.

2. service - to switch service over from toasted server a to healthy
server b, server a can hijack the ip and ethernet address of server b.
this can be done manually with a handy tool called fake (url below),
which can also be used mischievously, beware! (computer science
department can hijack the math department web server on april 1, etc.).
if you have time and resources to automate the whole thing further,
check out eddieware (url below), which is a complete suite of maybe-free
software to handle true load balancing and failover services. unlike the
other things i am mentioning here, i have not tried eddie yet.

> What do you need to do load sharing between multiple web servers?

this is simple and reasonable for static web sites. beware if you run
database applications or even cgi scripts -- you will have to evaluate
whether the backend database(s) can replicate in real time, whether cgi
can work from either server, and possibly make decisions to balance the
web service but park certain features on only one or the other server --
then look out, you just lost your redundancy :-)

if your library operates a nameserver (or someone operates one for you)
you can set up something called round-robin in the nameserver, which is
*not* load balancing (dns lb url below) but will help distribute queries
across two or more servers offering web or other types of service. this
is built into the nameserver and just takes a few minutes to configure
(nb getting the additional web servers up, running and ready is its own
independent headache :-)

genuine load balancing dns (where the nameserver monitors the load on
various servers and services and dispatches queries to the least-loaded
machine) is much trickier, i believe eddie (url below) includes an
attempt at such a load balancer. there are also hardware solutions to
implement load-balancing, the consulting group i work with purchased one
for a high-traffic client and the hardware never could be installed to
work properly :-(
 
> Currently we are using Netscape Enterprise Server running on a Sun Netra.
> We are debating whether to buy a second Sun, or whether we should move to
> NT and IIS.

platform decisions may depend more on the abilities and interests of
your network administrators than on the performance considerations. hope
you have have their buy-in no matter what you choose :-) if budget 
is important you might (re)consider running linux, freebsd, or 
openbsd on your sun/sparc/intel hardware.

btw i think a lot of the published ``research'' about hardware and
software performance is skewed, consider the nastily mis-conducted
research `study' of web server performance (nt vs. unix on matching but
misconfigured hardware) that microsoft commissioned mindcraft to
conduct. the trade press is in the process of debunking it this week
(url below).

keep in mind that you can also outsource just about *any* kind of web or
internet hosting need, to an ISP with 24/7 staff to monitor the systems
and act responsibly in the event of a problem. this may cost *less* and
result in *better* service (uptime, reliability, etc.) than bringing all
sorts of hardware and connectivity in-house, figuring it out, and then
reaching the right staff person when something goes awry with an obscure
box that no one else has ever touched.

URLs

CONTENT
rsync:   http://au2.samba.org/rsync/
cvs/web: http://durak.org/cvswebsites/

SERVICE
fake:    http://linux.zipworld.com.au/fake/
eddie:   http://eddieware.serc.rmit.edu.au/

LOAD BALANCE
eddie:   http://eddieware.serc.rmit.edu.au/
dns-lb:  http://www.isc.org/bind-lb.html

PLATFORM
so-called research:
http://www.zdnet.com/zdnn/stories/news/0,4586,2242246,00.html

--
mailto:sean at savvysearch.com                sean dreilinger, mlis
 http://www.savvysearch.com                http://durak.org/sean