North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Why do we use facilities with EPO's?

  • From: John C. A. Bambenek
  • Date: Wed Jul 25 14:53:44 2007
  • Dkim-signature: a=rsa-sha1; c=relaxed/relaxed;; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=CwGEQnKTYMQqLuJq05fSSVTrqakeYhp9HdtAr6lHDqD9qFYyCmJK1igdRSwKsbXFAePZqHeY44yr9J04qkdDQ29SsTXZPcdTu5voqpCpWF4SJtoxf3mmdB04iUx+9TUVBkG3cYV5SKrCf46KIXlZ1NIDRQBtdFKX3jrgvRjZxKg=
  • Domainkey-signature: a=rsa-sha1; c=nofws;; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=n/TSZ4NarTKSjo0srLxX1ioegsuRrZwwilV7eNNQcfqg6/insxXUn7eNnAWcYVqrKtu9XJX+5R1T+J1r0cMUN0wHuIBKqP0csUp+lR3CCv1xxmwVNb187NhPIu297c2uCKt46TA5qz2qUlQCw1KW3pDByOsu304HBB3Cu4frScg=

Funny story about that and the EPO we have here...

We have chilled water cooling in our server rooms.  A couple of years
ago we told the facilities guys there was sand in the lines.  They
didn't believe us.  This went back and forth for a few months until
the lines finally ground to a halt.  They admitted sand was in the

The bring out an HVAC guy... he closes the valve, opens the pipe,
nothing comes out.  He **opens** the valve, nothing comes out.  He
whacks on the pipes with a wrench, all the sand and lots of water come
out very fast.  By the time I got down there, the ceiling tiles were
drenched and looked more like sponges.  Half the room was soaked.

That would be a good reason to have an EPO right there. :)


On 7/25/07, Leo Bicknell <[email protected]> wrote:

I was complaining to some of the power designers during the building of a major facility that the EPO button represented a single point of failure, and effectively made all of the redundancy built into the power system useless. After all, what's the point of having two (or more) of anything, if there's one button somewhere that turns it all off?

What I found interesting is that a single EPO is not a hard and
fast rule.  They walked me through a twisty maze of the national
electric code, the national fire code, and local regulations.
Through that journey, they left me with a rather interesting tidbit.

The more "urban" an area the more likely it is to have strict fire
codes.  Typically these codes require a single EPO for the entire
structure, there's no way to compartmentalize to rooms or subsystems.
However in more rural areas this is often not so, and they had in
fact built data centers to code WITHOUT a single building EPO in
several locations.  That's to say there was no EPO, but that it may
only affect a single room, or even a single device.

If they can be avoided, why do we put up with them?  Do we really
want our colo in downtown San Francisco bad enough to take the risk
of having a single point of failure?  How can we, as engineers, ask
questions about how many generators, how much fuel, and yet take
for granted that there is one button on the wall that makes it all
turn off?  Is it simply that having colo in the middle of the city
is so convenient that it overrides the increased cost and the reduced
redundancy that are necessitated by that location?

       Leo Bicknell - [email protected] - CCIE 3440
        PGP keys at
Read TMBG List - [email protected],

/* [email protected] is an alias for all ISC handlers.
   Please include the list in all replies to keep everyone informed.
   You may receive more than one response */