The segmentation of data centers into alternating hot and cold aisles is an established best practice. A number of manufacturers are taking this premise of airflow separation a step further by marketing "containment" solutions. By containing the hot or cold aisle, the air paths have little chance to mix, presenting data center operators with both reliability and efficiency gains.
To view the recording of the webinar presentation, please visit http://www.42u.com/webinars/Aisle-Containment-Webinar/playback.htm
8. Thermal Imaging Thermal Maps show where there are hot and cold spots within the data center, indicating areas of potential risk. Based on sensors placed at the top, middle and bottom of the instrumented racks.
9. Baseline C H H C No Containment High Degree of Air Mixing High Inlet Temperatures Thermal Imaging – Baseline
13. After Containment C H H C Cold Aisles Contained No Change to CRAHs Overcooled CAC: Existing Data Center
14. Existing Data Center- With Control With Control C H H C Controlled CRAHs with possible higher set points, Lower Fan Speeds and Increased Water Supply Temperatures ASHRAE Inlet Temperatures Increased Return Temperatures
29. An Efficiency Study Process Report Remediate Measure Benchmark Data Center Survey Best practices & ROI Potential rebates Executive managerial buy-in Establish efficiency team & goals Plan Determine methodology Install instrumentation & software Perform energy utilization audit Calculate current benchmark Data Collection Environmental Monitoring Analysis Recommendations Reconfiguration plan Efficiency improvements Facilities Data Infrastructure Data IT Equipment Data Environmental Conditions Design and limitations Efficiency improvements
30.
31. Q&A For a copy of today’s presentation please email [email_address] For additional information on 42U’s products and services, please visit: http://www.42U.com
Today our objective will be to introduce and explain some of the available containment solutions in the marketplace. Containment is an intriguing yet simple approach. A lot of data center cooling strategies have been around for a long time: raised floors, hot aisle/cold aisle, perimeter cooling units. Containment is not brand new per se, but I think it’s fair to say that that only recently has it gained a lot of attention. So, we’re eager to discuss it here today.
Here is our agenda. It’s straightforward. We’ll discuss briefly some airflow management strategies. We’ll jump into aisle containment strategies, both cold aisle and hot aisle. We’ll look at some of the considerations for deploying both of those approaches. Then we’ll present a brief containment case study, where 42U provided the turnkey service of specifying, selling, and supporting the containment solution. Finally, we’ll conclude with some Q&A from those of you in the audience. So, again, thank you for attending. We appreciate it. And with that we’ll jump into the meat of the presentation.
This quote comes from a great NYT article on data centers, published in June of this year. For those of us who work in this space, there’s no shortage of conversations about data centers and energy. You can’t go to a data center event or tradeshow without hearing about “green data centers” or “data center efficiency.” Most of us have heard time and time again about the EPA’s report: consumption in the billions of kilowatt hours; cost in the billions of dollars—very large numbers. There have notable cases where large, enterprise data centers in the US have exhausted their utility supply with no recourse to get more. If you design a new data center with efficiency in mind, and you have a multi-megawatt type demand, PG&E, a Silicon Valley utility, has cut six figure checks for end users like Network Appliance and HP.
Why are data centers power hungry? To articulate answers to that complex question, we could look at a couple of these areas. Server utilization : The Uptime Institute recently projected that up to 30% of servers in a typical deployment may be comatose. Those servers may consume up to 60% of their max power draw while doing nothing. With mission criticality, there can be over provisioning . You also have (2) dissimilar groups—IT and Facilities—who may talk, who may not, who may have different objectives Finally, we have to consider density and demand. The Gartner slide shows where a fully-loaded rack is headed. Granted this is not the case for every server environment, but it’s possible. 42U Experience Customers are running 60Amp, 3 phase to server cabinets We have customers who are planning for 30kW at the rack level We have customers deploying “liquid cooling” at the rack
So, demand grows. And availability and performance are sacred. We know this. But even so, no one is immune to what’s happening in the economy. And as we know, right now in the outside world it’s not great. The right hand side of this chart shows how the current economic climate relates to the data center. Server shipments in Q1 were down significantly year over year. And, a lot of users are coping with smaller budgets and possibly a delay in planned data center projects. However, demand is still going to be there. If a data center project is postponed, we may have to make a legacy facility last longer, and we may experience pressure both within and outside our organizations to become more “green”. So, part of this diagram we cannot control. We cannot control the ebb and flow of the economy. And we can’t control to an extent what happens to our budgets. But the left side of the diagram, as we highlight here, shows some things we have control of. If an expansion has been delayed, there are steps we can take to prolong the useful life of our facilities, delay capital expenses, and become more efficient at the same time.
Containment is an area that addresses both efficiency and capacity. But, the crux of containment is airflow management. Hot aisle/cold aisle is a best practice, and the basis for containment. We can see from our simple diagram the “conventional” data centers- underfloor plenum, hot aisle cold aisle, an open ceiling return. But even with airflow separation, as we see here, there are still opportunities for the air paths to mix- above cabinets and and at the end of rows. If this mixing is prolonged, we know the story: hot spots, reduced reliability. But airflow is uniquely challenging because it’s difficult to manage, control, and optimize something you cannot see.
There are ways to capture a snapshot of airflow, but they do have their limitations…… CFD is an expensive but popular option. From the picture, we can see a color coded map, noting a large amount of air mixing. There are some caveats to CFD. Cooling equipment doesn’t always perform to spec. The data entered into the software program could miss some key obstructions. And the model itself could take considerable time to create.
For those reasons, some of our customers prefer an on-site real time monitoring solution. We can see an example of that here. Room level Thermal Imaging Lots of assumptions being made here as the temp probes are located at top, middle, bottom of front and back of every 3 rd rack Not a completely accurate picture of the entire room
(drilling down to the row level) This image comes from the real-time thermal imaging. We can see a high degree of air mixing and some rack areas which are subjected to high inlet temperatures. And we are reminded here of the continual challenge of data center cooling: providing the right amount of air at the right temperature in the right place at the right time. Air mixing compromises this objective. We subject the equipment to elevated inlet temperatures and chances are, we’re moving more air than we need to, resulting in unnecessary consumption and cost.
Containment is an effective strategy to eliminate mixing. As seen here and the following slide, we have options in cold aisle containment.
There are certainly considerations that need to be accounted for when determining which aisle containment solution to explore…… With cold aisle containment, we ensure that the chilled air does productive cooling. Chilled air is directed toward the servers; hot air disperses throughout the room. Assuming that we’ve implemented simple best practices (blanking panels in all unused spaces and all holes filled with grommets), we should have no air mixing. Cold aisle containment can be a greenfield or brownfield project.
Some of the advantages of cold aisle containment are noted here. Though there are a variety of important temperatures throughout the data center, the most important temperature is right in front of the servers. The only ASHRAE standard exists there. So, we’re containing the airflow that matters most to the comfort of the servers. The approach also addresses the prevalence of existing raised floor environments. Despite its efficiency gains, the InRow air conditioner is itself a newer product and not as established as traditional cooling units. Cold Aisle Containment is more retrofit friendly and we can accomplish the joint objective of increasing capacity and becoming more efficient.
On this slide, we revert to our previous thermal imaging after cold aisle containment was implemented. The mixing has been eliminated and we have an even temperature distribution. By the color coded map, the cold aisles are cold…very cold. The air handlers are still running at their previous settings before containment and are moving more air than is necessary.
Taking the newfound capacity and implementing control, we have this picture. We’re no longer in the low 60s and high 50s. We’re in the low to mid 70s. The cold aisles are comfortable within the ASHRAE range. The hot aisles are hotter, as they should be. We should see efficiencies at the CRAC level with the warmer return temperatures. The CRACs themselves have a higher temperature setpoint and regulated fan speeds.
Slide 12, on the other hand, shows us hot aisle containment. The example above includes modular in-row cooling units in contrast to the traditional bulky perimeter units. The close proximity of the in-row AC to the load ensures that the cooling coil sees very warm return temperatures; there’s no bypass air, which allows us to maximize the efficiency of that coil. Though it’s not pictured here, we can also accomplish hot aisle containment using drop ceilings and “chimney cabinets,” where all server exhaust is channeled into the ceiling, and directed back to the CRAC return.
Here we delve into hot aisle containment. Our containment case study focuses on hot aisle containment, so we’ll discuss in more detail later on. To the high-level advantages, hot aisle containment aims to make the rest of the data center more comfortable. The hot air is contained, meaning the rest of the room feels like the server inlet temperature—a comfortable 70 degrees or so. Additionally, in the event of a cooling failure, the full room volume of air is available to cool the servers. This might provide a couple of additional minutes for operators to take corrective actions or to shutdown equipment gracefully. Furthermore, we’re capturing server exhaust air at its warmest point, which, as we discussed previously, enhances the performance of the cooling coils in our air conditioning units.
But we can accomplish containment with what we term a “curtain” approach. We can do hot aisle or cold aisle containment with this product. For cold aisle containment to the left, you’ll see strip doors, similar to clean rooms or refrigerated rooms, at the end of the aisle. Overtop, you’ll see a roof made of the same material. For hot aisle containment to the right, you’ll see the same strip doors and no roof. The hot air must be removed from the room, so this particular installation uses a ceiling return to route the hot air back into the building system.
The manufacturers of both solutions will tout the superiority of their product. But, the fact remains that there are shared advantages for both flavors of containment. Both eliminate mixing Both allow you to tune airflow. If there’s no mixing or less air lost to bypass, we don’t need to supply excess air to meet the load. We can make the CFM demanded by IT equipment match the CFM supplied by the air handlers Both promote warmer return temperatures to the air conditioners Both provide better supply-side temperature control So, they both work. They both enhance the predictability and efficiency of data center cooling topologies. And they are an upgrade over conventional raised floors.
What needs to be considered when selecting your aisle containment approach? The wild card in the containment debate is your data center. There is rarely anything that is one size fits all; every room, environment, application has its nuances as the pictures show. The middle picture serves a dual purpose of data center and storage closet. So, the type of containment solution that you can deploy is very dependent on your variables. Some of those considerations are noted here: Uniformity of racks and aisles? DIFFERENT HEIGHTS, WIDTHS? Rack manufacturer offer containment solution? Ceiling plenum for hot aisle containment? Can you use it if you’re in a leased space? Raised floor clear or full of obstructions?
Here is a case study of one of our clients we recently worked with through the implementation of hot aisle containment. Main challenges include a move to high density server racks with the addition of blade chassis and 1U servers Not completely uniform in hot/cold aisle arrangement No raised floor, all cooling and heat removal was done overhead Could not adequately prevent air mixing using rear door exhaust units, temporary industrial fans, and temporary “roll away” AC units
Because of the curtain approach, we were able to customize a containment solution to fit their environment. Hot containment, curtains extend to ceiling, used existing drop ceiling for hot air plenum
Because of the curtain approach, we were able to customize a containment solution to fit their environment. Hot containment, curtains extend to ceiling, used existing drop ceiling for hot air plenum
Key Benefits: 21 rack air removal units turned down to their minimum CFM, an overall decrease of over 33K CFM Exhaust fans were turned off and removed from room (4) portable AC units are now simply cycling air with no demand on the compressor Eliminated hot spots, produced stable inlet temperatures w/in ASHRAE standards Extra Benefits: Lab environment was much cooler and less noise
That was a good example of a retro fit using curtains, but there are more progressive ways to implement containment to gain efficiencies……… A lot of our focus so far has been containment in retrofit applications. But it’s worth noting that professionals are exploring containment for new builds. This slide comes from an EDS white paper, written on cold aisle containment. During their testing, with a conventional room layout: 24 inch raised floor, Open ceiling return Hot aisle, cold aisle arrangement 4 foot wide cold aisle (3) DX 20-ton CRAC units, 10 server racks They found, with cold aisle containment, they could support up to 17kW per rack They then apply these findings to a new data center design/build. The left hand picture shows the conventional data center with 3kW per rack and the CAC data center at 17kW per rack. At a total load of 306kW, the overall floor space is compressed from 3000 sq ft to 900 sq ft. The number of air handlers is reduced. Floorstanding power distribution is simplified. There’s a smaller real estate envelope all around and should make the management of that space simpler.
Continuing on the subject of new builds… There are a few cooling products we can bring into containment. A couple of solutions, seen here, bring the source of cooling directly to the heat generating source: the server rack. The solution on the LH side brings an In-row air conditioner into the cold aisle containment structure. The solution on the RH side, though difficult to see, uses an overhead cooling unit also installed in the cold aisle containment structure. One of the advantages here, for newer builds, is fan energy. The distance between the air conditioner and the load is reduced considerably
This particular cooling approach brings both strategies together: both hot aisle and cold aisle containment in the same rack footprint. The product completely isolates the compute load from the room. Here we are immune to any airflow quirks. Cold air has no choice but to enter the servers. Hot air has no choice but to pass through the heat exchanger. There are additional efficiencies outside the scope of this presentation. But it builds on the same containment principle.
Revisit the same slide. This is what data center professionals are facing. Reiterate the “do more with less” slogan. Where can you turn?
The 42U process and value proposition. Through this process, we can determine the most relevant ways to streamline your unique environment. Is it containment? Is it low-cost, no cost changes? We can determine everything through this cyclical process.
Based on these considerations, we can make a few statements on the relevance of containment solutions: Steel containment is more aesthetic but better suited for new builds At least new rows Some manufacturers offer “steel” solutions to take into account different cabinets and different dimensions. Hot Aisle Containment, other than curtain solutions, can require extensive rework to deploy in existing cabinet rows InRow Units: new mechanical infrastructure, design, engineering Chimney cabinets: new solid rear doors, new roofs, chimney apparatus, CRAC extensions Curtain solutions, though not beautiful, offer the most flexibility Cold Aisle or Hot Aisle Any combination of rack manufacturers Inexpensive; quickly deployed To use the low-hanging fruit cliché, containment is one of those areas. Eliminating air mixing has a positive impact on your environment.
Thank you for your time and attention. We will now open up the floor for questions.