North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: rack power question

  • From: Justin Shore
  • Date: Sun Mar 23 00:21:10 2008


This greatly depends on what you want to do with the space. If you're putting in co-lo space by the square footage footprint then your requirements will be much less. If you expect a large percentage of it to be leased out to an enterprise then you should expect the customers to use every last U in a cabinet before leasing the next cabinet in the row. Ie your power usage will be immense.


I did something similar about 2 years ago. We were moving a customer from one facility to another. We mapped out each cabinet including server models. I looked up maximum power consumption for each model including startup consumption. The heaviest loaded cabinet specced out at 12,000w. The cabinet was full of old 1U servers. New 1U servers are the worst-case scenario by far. 12k is rather low IMHO. Some industry analysts estimate that the power requirements for high-density applications scale as high as 40kw.

http://www.servertechblog.com/pages/2007/01/cabinet_level_p.html

There are a few things to remember. Code only permits you to load a circuit to 80% of its maximum-rated capacity. The remaining 20% is the safety margin required by the NEC. Knowing this that means that the 12Kw specified above require 7x 20a 120v circuits or 5x 30a 120v circuits.

You can get 20a and 30a horizontal PDUs for both 120v and 240v. There are also 208v options. You can also get up to 40a vertical PDUs. One word of caution about the vertical PDUs. If your cabinets aren't deep enough in the rear (think J Lo) the power cabling will get in the way of the rails and other server cabling. There are others but they are less common.

Also remember that many of the larger servers (such as the Dell 6850s or 6950s) are 240v and will require a pair of dedicated circuits (20a or 30a).

I would also recommend that you look into in-row power distribution cabinets like the Liebert FDC. This means shorter home-runs for the large number of circuits you'll be putting in (saving your a bundle in copper too). It also means less under-floor wiring to work around, making future changes much easier. Changes in distribution cabinets are also much easier, safer and less prone to accidents/mistakes than they are in distribution panels.

Grounding is a topic that is worthy of its own book. Consult an electrician used to working with data centers. Don't overlook this critical thing. Standby power sources fall into this topic as well. How many 3-phase generators are you going to need to keep your UPSs hot?

I'm curious what your cooling plans are. I would encourage you to consider geothermal cooling though. The efficiencies that geothermal brings to the table are worth you time to investigate.

Best of luck,
 Justin


Patrick Giagnocavo wrote:

Hopefully this classifies as on-topic...


I am discussing with some investors the possible setup of new datacenter space.

Obviously we want to be able to fill a rack, and in order to do so, we need to provide enough power to each rack.

Right now we are in spreadsheet mode evaluating different scenarios.

Are there cases where more than 6000W per rack would be needed?

(We are not worried about cooling due to the special circumstances of the space.)

Would someone pay extra for > 7KW in a rack? What would be the maximum you could ever see yourself needing in order to power all 42U ?

Cordially

Patrick Giagnocavo
[email protected]