The topic of raising temperatures in data centers used to be met with much criticism in the industry, but in recent years has become more accepted. A big driver for this acceptance has been ASHRAE’s expanded envelope for recommended and allowable server inlet temperatures. However, while this has eased the discussion, there are still some questions that have been left unanswered. What’s the difference between recommended and allowable? Which one is best to use? What steps must be taken to safely raise set points? How do you ensure servers are still adequately cooled? What if you have different server types (A1, A2, A3, A4)? This presentation will examine these questions to give a clearer understanding of ASHRAE’s recommended and allowable guidelines. Also covered will be an explanation on how, in some cases, it is possible to raise cooling control set points without raising server inlet temperatures.

1. August 24-27, 2020
30519_DCW20

2. datacenterworld.com
Clarifying ASHRAE's Recommended
vs. Allowable Temperature Envelopes
and How to Raise Cooling Set Points
Without Raising Inlet Temperatures
Lars Strong, P.E.
Senior Engineer and Company Science
Officer
Upsite Technologies

3. datacenterworld.com
Speaker Background
Lars Strong, Senior Engineer and Company Science Officer, Upsite
Technologies
• Thought leader and recognized expert on data center airflow management
and cooling optimization with over 23 years of experience
• Certified U.S. Department of Energy Data Center Energy Practitioner (DCEP)
HVAC and IT Specialist
• Previous AFCOM presentations:
• For Most Data Centers, Liquid and Air Cooling Will Not be Mutually
Exclusive
• How IT Decisions Impact Facilities: The Benefit of Mutual Understanding
• Designing, Deploying, and Managing Efficient Data Centers
• Data Center Cooling Efficiency: Understanding the Science of the 4
Delta T’s

4. datacenterworld.com
Agenda
• Background
• Who is ASHRAE?
• ASHRAE Thermal Guidelines
• IT Equipment Intake Temperatures
• What’s the Difference Between Recommended and Allowable?
• The “X” Factor
• Which One Should You Use?
• How to Raise Cooling Set Points Without Raising Inlet Temperatures
• The Role of Airflow Management
• How to Raise Temperatures Safely

5. datacenterworld.com
Who is ASHRAE?
• Founded in 1894, ASHRAE is the American Society of Heating, Refrigerating and Air-Conditioning Engineers
• In 1999 a group of thermal engineers from different IT manufacturing companies formed a thermal
management consortium. This consortium evolved into ASHRAE Technical Committee (TC9.9)
• TC 9.9 developed Thermal Guidelines for Data Processing Environments, 1st addition 2004. Prior to the first
edition, there was no single source in the data center industry for ITE temperature and humidity requirements
• Includes guidelines for dew point temperature, dry bulb temperature, wet bulb temperature, and relative
humidity
• Their guidelines have been a topic of much debate throughout the industry
• ASHRAE guidelines are the most widely accepted guidelines globally

6. datacenterworld.com
ASHRAE Thermal Guidelines

7. datacenterworld.com
IT Equipment Intake Temperatures
• IT equipment intake temperatures is what we are mostly concerned about
• Temperatures vary by server class
• A1 to A4 is the most common, though A1 is almost completely phased out

8. datacenterworld.com
What’s the Difference Between Recommended and
Allowable?
• The first, and probably least understood difference
between the recommended limits and the allowable limits is
that the recommended envelope is basically an arbitrary
number arrived at through a mostly consensus-reaching
process among the TC9.9 committee members.
• Presented as a “guideline”, not an enforceable
standard
• The allowable range is a much less arbitrary number
closely approximating the user documentation specifications
from the IT equipment OEMs warranty limits

9. datacenterworld.com
What’s the Difference Between Recommended and
Allowable?
• Another difference between the recommended envelope
and allowable envelope has to do with server reliability –
superficially, IT equipment operating within the recommended
envelope is going to last longer and have fewer failures than
IT equipment operating in the allowable envelope
• The biggest difference between recommended temperature
limits and allowable temperature limits is the cost of
designing, building, and operating the data center

10. datacenterworld.com
The “X” Factor
• After causing some confusion in the industry with the original release of the allowable ranges for server inlet
temperatures that could be permitted for some undefined short periods of time, ASHRAE clarified matters some
seven years ago by developing the “X” factor
• The “X” factor is meant to help organizations determine the impact on reliability that results from periods of
time operating in the wider range of “allowable temperatures”

11. datacenterworld.com
The “X” Factor
• The server OEMs on TC9.9 would not share specific information about their product’s failure rates, but they
did share information on variances of failure rates based on inlet temperatures
• They eventually agreed on 68˚F as the baseline figure (X Factor = 1), and then agreed on what increases or
decreases in that base failure rate could be expected based on server inlet temperatures being either higher or
lower than 68˚F
• For example, at 59˚F (X Factor = 0.72) we should expect to see about 72% of the failures we might expect to
see at 68˚F and at 77˚F (X Factor = 1.24) we might expect to see 124% of the failures we might typically see at
68˚F

12. datacenterworld.com
Which Should You Use?
• In terms of server reliability, operating within the allowable envelope is the most common decision
for most facilities
• For facilities that aren’t as concerned about reliability due to quicker IT refresh cycles, operating
within the allowable range could be more desirable

13. datacenterworld.com
Informed Decision Considerations
• Turnover of IT equipment
• Infrastructure configurations
• Geographical location
• Economization (airside, water side)
• IT equipment redundancy
• SLAs

14. datacenterworld.com
Raising Temperature Set Points
• Raising temperature set points used to be a taboo subject, but has gained traction over the last few years as a
way to improve efficiency and increase capacity
• While a warmer data center does have it’s benefits, the process of doing so requires more than merely turning
up the thermostat
• Even with a rather typical return air set point of 72⁰F (resulting in supply air being delivered around 54⁰F) many
data centers still require a significant over-supply of cool air volume to maintain all the servers in that space with
inlet temperatures below 80⁰F
• The cause is poor airflow management, therefore the process should start with airflow management

15. datacenterworld.com
The Role of Airflow Management
• In a standard data center environment that is cooled by air, airflow management is simply the process of
managing the flow of air through the room
• Specifically, managing the flow of conditioned or cooled air to the intakes of IT equipment and the flow of
exhaust air back to the cooling units
• There are many configurations for IT equipment layout and cooling system designs, in all cases airflow
management is simply preventing the hot and cold airstreams from mixing
• On its own, improving airflow management can reduce IT equipment intake air temperatures which improves
IT equipment reliability
• It also enables the adjustment of cooling controls and set points to reduce operating costs, improve cooling
capacity, and defer capital expenditure

16. datacenterworld.com
Ideal Hot and Cold Aisle Configuration (With Good Airflow
Management)

17. datacenterworld.com
Typical Hot and Cold Aisle Configuration (With Poor Airflow Management)

18. datacenterworld.com
How Airflow Management Relates to PUE
• There is a direct relationship between airflow
management and PUE – the better the airflow
management, the better the PUE
• Mechanical cooling + fan cooling is 35% of total
load and 73% of non-IT load
Total Load
IT Load
PUE = = 1.92

19. datacenterworld.com
The Goal of Airflow Management and Cooling
Optimization
• Provide appropriate IT equipment intake air conditions and redundant cooling capacity, with the lowest
possible flow rate of conditioned air at the warmest possible temperature

20. datacenterworld.com
The Four Delta T’s (ΔT)
1. Though IT equipment
2. IT equipment exhaust to cooling unit
3. Through cooling unit
4. Cooling unit supply to IT equipment intake

21. datacenterworld.com
The Four Delta T’s (ΔT)
• In most data centers the greatest loss of
efficiency is having a low temperature of air
returning to the cooling units
• The goal is to always have the highest
possible return air temperature to cooling units

22. datacenterworld.com
What is Cooling Optimization?
• Cooling optimization is the process of making adjustments to the cooling system controls to improve energy
efficiency resulting in reduced operating costs, improve cooling capacity, improve IT equipment reliability, and
defer capital expenditure
• Cooling optimization is often an iterative process of making adjustments to controls, allowing the system to
equalize, and then making additional adjustments, and so on

23. datacenterworld.com
Benefits of Raising Temperature Set Points
• By implementing airflow management best practices for the
raised floor, rack, and row, IT intake air temperatures become
lower and more even
• Cooling unit set points can then be raised without
increasing the maximum IT intake air temperature
• Higher return air temperatures result in increased cooling
capacity and improved efficiency

24. datacenterworld.com
Benefits of Raising Temperature Set Points –
Example

25. datacenterworld.com
Steps to Safely Raise Temperature Set Points
• Step 1: Survey all your IT equipment intake air temperatures
• Step 2: Implement airflow management initiatives at the Rack, Raised Floor, and Row levels to make the IT
equipment intake air temperatures as low and even as possible
• Hot, or the warmest, spots are an indication of exhaust air recirculation. Find the source of recirculation
and block it.
• Step 3: Raise your cooling-unit set points
• Step 4: Reduce your cooling-unit fan speeds, or turn off cooling units without VFD
• Throughout the process always monitor IT equipment intake air temperatures for the potential formation of hot
spots

26. datacenterworld.com
Key Takeaways
• ASHRAE’s recommended intake air temperatures are presented as “guidelines,” without the enforceability of
code or stature of standards, regardless of the industry’s acceptance of them as default standards
• The allowable limits have an enforceability element lacking in the recommended limit: IT equipment OEM
warranty limits
• Operating in the allowable range can have profound financial benefits
• Improving airflow management is fundamental to increasing efficiency, cooling capacity, and maximizing the
ability to raise cooling unit set points without raising IT equipment intake air temperatures

27. datacenterworld.com
Questions?
Lars Strong
Upsite Technologies
lds@upsite.com
505.699.9069
Facebook LinkedInTwitter
InstagramYouTube Upsite Blog