So true. I took two looks at the APC's & generator that I have. As well as
the power part of the grand scheme of things, I wrote a scrip to take the
data sent to the backup server and replicate it to another server. Today, I
spent time looking over our AC units with the thought in mind of "What If?".
"How long would it take me to get my messily 10 servers back on line while
having the AC going?". Here in Vegas - as I'm sure it is in Huston & Dallas
- the temperatures are starting climb and I wouldn't last long without solid
AC.
Doug
-----Original Message-----
From: Chris Gebhardt - VIRTBIZ Internet [mailto:cobaltfacts (at mark) virtbiz.com]
Sent: Sunday, June 01, 2008 4:47 PM
To: coba-e (at mark) bluequartz.org
Subject: [coba-e:13133] Re: Somewhat OT: Major Outage (at mark) ThePlanet.com
Doug Harvey wrote:
> One thing's for sure. It makes you take a second look at whatever setup
> someone currently has and provokes thought about future plans.
You are right, Doug.
As a company that provides straight webhosting, dedicated servers, and
colo, rest assured that we question ourselves daily. Which probably
doesn't set us that far apart from most of the providers - at least, not
those that I would consider to be worthy "competitors".
I think the big thing to realize is that apart from monetary resources
(some companies do more with more, some do more with less) it's all
about the human elements. As I understand this particular failure
(given my conversation w/ colleagues and folks privy to the situation on
the ground in Houston) it's clear that, once again, all the equipment,
systems, redundancies, and strategies are *so close* to being completely
out the window when the human element is involved.
Any provider that tells you a failure cannot strike and they can prevent
any circumstance from causing disaster is either lying or sadly misinformed.
Unfortunately, when you're dealing with such scale as this (over 9000
servers) it's flat impossible expect the provider to be agile enough for
fast resolution.
It would not be appropriate for me to provide a lot of commentary on the
particular choices made by the provider in question here.
The better use of the 1's and 0's, if I can give any input at all, would
be to take this opportunity to look at your backup plan.
As a BlueQuartz user, with reasonable backups you could typically avoid
a days-long outage like this without much trouble at all.
Food for thought...
--
Chris Gebhardt
VIRTBIZ Internet Services
Access, Web Hosting, Colocation, Dedicated
www.virtbiz.com | toll-free (866) 4 VIRTBIZ