This document summarizes a presentation about how IT decisions impact facilities management in data centers. It discusses how choices around server type, server class, equipment airflow patterns, storage type, cooling unit setpoints, cage layout, and airflow management best practices can significantly affect operating costs, capacity, and mechanical systems. Selecting servers with higher delta T or blade servers instead of pizza boxes, for example, can substantially reduce data center cooling needs.

1. How IT Decisions Impact Facilities: The
Importance of Mutual Understanding
Lars Strong, P.E., Upsite Technologies
Ian Seaton
Airflow Management Awareness Month
June 15, 2016

2. Lars is a thought leader and
recognized expert on data center
optimization. He currently serves as
the Senior Engineer and Company
Science Officer of Upsite
Technologies. Lars is a certified U.S.
Department of Energy Data Center
Energy Practitioner (DCEP) HVAC
Specialist.
6/16/2016 © Upsite Technologies, Inc. 2
Speaker Bios
Ian has over 30 years of electro-mechanical
product development experience. He serves
on the BICSI 002 Data Center Standard
subcommittee as a working group leader and
editor, and he contributes to the Green Grid
Airflow Management Energy Calculator
Committee. He recently retired as a
corresponding member of ASHRAE TC9.9,
and he is a patent holder for airflow
performance algorithms.
Lars Strong Ian Seaton

3.  Specifying Server Type: Cooling Delta T (ΔT)
 Specifying Server Class: Class (A1, A2, A3, or a4)
 Specifying Equipment That Breaths Front to Back
 Specifying Storage Type: Solid State or Tape
 Specifying Cooling Unit Set Point
 Specifying Cages/Layout That is Compatible with
Containment
 Airflow Management Best Practices
6/16/2016 © Upsite Technologies, Inc. 3
Agenda

4. The Importance of Mutual Understanding
 Decisions and actions typically under the
jurisdiction of the IT side of data center
management can have a profound impact on
the mechanical systems, operating costs, and
capacity of the data center.
 By understanding these impacts, IT and
facilities management are able to develop a
cooperative approach to managing the data
center.
How IT Decisions Impact Facilities
6/16/2016 © Upsite Technologies, Inc. 4

5. SPECIFYING SERVER TYPE: COOLING
DELTA T (ΔT)
© Upsite Technologies, Inc.6/16/2016 5

6. ΔT Through IT Equipment
 Cooling unit operating expense
 Cooling unit acquisition capital expense
Decisions About Server Types Affect:
6/16/2016 © Upsite Technologies, Inc. 6

7. Heat Transfer Equation
CFM =
CFM = cubic feet per minute of airflow through the
server
3.16 = factor for density of air at sea level in relation to
⁰F
ΔT = temperature rise of air through the server in ⁰F
3.16 x W
Delta T
IT Equipment Required Flow Rate
IT Equipment Delta T (⁰F)
15 20 25 30 35 40
Required
flow rate
(CFM/kW)
211 158 126 105 90 79
ΔT Through IT Equipment
6/16/2016 © Upsite Technologies, Inc. 7

8. • “Pizza box” servers
• at 20 ⁰F ΔT consume 158 CFM / kW
• “Blade” servers
• at 35 ⁰F ΔT consume 90 CFM / kW
ΔT Through IT Equipment
6/16/2016 © Upsite Technologies, Inc. 8

9. • 500kW of “Pizza box” servers
• 1,000 servers
• Would require 79,000 CFM of chilled air
• 500kW of “Blade” servers
• 1,600 servers (100 chassis)
• Would require 45,140 CFM of chilled air
ΔT Through IT Equipment
6/16/2016 © Upsite Technologies, Inc. 9

10. • “Blade” servers (35 ⁰F ΔT)
• Servers require 43% less airflow rate
• Cooling unit fans require 81% less energy
• At 50% fan speed
• 50% airflow rate
• Only 12.5% of fan energy (87.5% reduction)
ΔT Through IT Equipment
6/16/2016 © Upsite Technologies, Inc. 10

11. SPECIFYING SERVER CLASS: CLASS
(A1, A2, A3, OR A4)
© Upsite Technologies, Inc.6/16/2016 11

12. • Access to free cooling hours
• Chiller operating efficiency
• Capital investment for the mechanical plant
• PUE anomalies
• Warranty and associated reliability costs
• Building footprint and real estate investment
ASHRAE Server Class
Decisions About ASHRAE Server Class Affect:
6/16/2016 © Upsite Technologies, Inc. 12

13. ASHRAE Environmental Guidelines
6/16/2016 © Upsite Technologies, Inc. 13

14. ASHRAE Server Class
How to know the effect of temperature on ICT equipment
reliability/life?
How to know how long it is OK to operate in allowable
temperature range?
The “X” Factor
1. Baseline = 24/7 operation @ 68˚F server inlet temperature
2. OEM historical data on user failure reports
3. Performance variations above, at and below baseline 68˚
4. Premise is that the data center temperature follows mother nature

15. ASHRAE Server Class
Time at temperature weighted failure rate
calculations for IT equipment in Chicago
Inlet Temperature The “X” factor % of Hours
59⁰F ≤ T ≤ 68⁰F .865
68⁰F ≤ T ≤ 77⁰F 1.13
77⁰F ≤ T ≤ 86⁰F 1.335
86⁰F ≤ T ≤ 95⁰F 1.482
67.6%
17.2%
10.6%
4.6%
Net X-Factor = 0.99
6/16/2016 © Upsite Technologies, Inc. 15

16. ASHRAE Server Class
Time-weighted X-factor estimates
for air-side economizer for selected U.S. cities
versus baseline 68˚ for 8,760 hours
Improved Failure Rates
San Francisco 90%
Seattle 90%
Boston 96%
Denver 97%
Los Angeles 98%
Chicago 99%
Worse Failure Rates
Miami 128%
Phoenix 120%
Houston 113%
Dallas 110%
Atlanta 104%
Washington DC 101%
6/16/2016 © Upsite Technologies, Inc. 16

17. SPECIFYING EQUIPMENT THAT
BREATHES FRONT TO BACK
© Upsite Technologies, Inc.6/16/2016 17

18. • Integrity of hot aisle separation
• Potential floor density thresholds
• Fan and temperature set points
• Integration of switches and servers
Specifying Equipment
Decisions on ICT Equipment Breathing Patterns Affect:
6/16/2016 © Upsite Technologies, Inc. 18

19. • As far as the mechanical health of the data center, all
equipment should be:
• rack-mountable
• Breathe from front to rear
• Recognized best practice of sources as diverse as:
• European Code of Conduct for Data Centers
• BICSI-002
• ASHRAE TC9.9
Specifying Equipment
6/16/2016 © Upsite Technologies, Inc. 19

20. • CFD model – switch flow
side to side
• 3,500 sq. ft., 514 kW
• 3kW rack with 73 ⁰F to 90
⁰F intake air temperatures
• 6kW switch rack ingesting
56 ⁰F to 85 ⁰F air
• Allowable by manufacturer
but exceeded internal SLA
• 72 ⁰F return set point,
resulting in 54 ⁰F supply
• 82,500 CFM cooling
supply for 67,900 CFM IT
demand
Specifying Equipment
6/16/2016 © Upsite Technologies, Inc. 20

21. • CFD model – switch flow
front to back
• 3kW rack with 75 ⁰F to 76
⁰F intake air temperatures
• 6kW switch rack ingesting
75 ⁰F to 76 ⁰F air
• Supply temperature
increased from 54 ⁰F to 75
⁰F
• 38% chiller plant energy
savings
• cooling supply reduced
from 82,500 CFM to
72,000 CFM
• 33.5% CRAH fan energy
savings
Specifying Equipment
6/16/2016 © Upsite Technologies, Inc. 21

22. SPECIFYING STORAGE TYPE: SOLID
STATE OR TAPE
© Upsite Technologies, Inc.6/16/2016 22

23. • Access to free cooling
• Total operating energy budget
• Mechanical plant acquisition and construction costs
• Humidity management investment
Solid State vs Tape Storage
Decisions on Solid State vs Tape Storage Affect:
6/16/2016 © Upsite Technologies, Inc. 23

24. • ASHRAE TC9.9 rate of temperature change
boundaries:
• Tape storage - 9°F per hour
• Solid state storage - 36°F per hour
• Weigh solid state storage premium against
cost avoidance of acquiring, constructing and
operating a mechanical plant
Solid State vs Tape Storage
6/16/2016 © Upsite Technologies, Inc. 24

25. SPECIFYING COOLING UNIT SET
POINT
© Upsite Technologies, Inc.6/16/2016 25

26. • Chiller efficiency
• Cooling unit efficiency and life (hot & cold cycling)
• Humidity management operating expenses
• Access to free cooling
• General operational discipline
Specifying Cooling Unit Set Point
Decisions Specifying Cooling Unit Set Point Affect:
6/16/2016 © Upsite Technologies, Inc. 26

27. • Comfort cooling ≠ data center cooling
• It is still common to see 72° F cooling unit return set
today, and lower set points in the range of 68° F are
still not uncommon
• Often driven by IT conservatism or hot spot
mitigation
Specifying Cooling Unit Set Point
6/16/2016 © Upsite Technologies, Inc. 27

28. • Cooling units often cool the air 18° F, resulting in
supply temperatures as low as 50° F, potentially
causing dew point problems.
• At 50° F, 100% RH or condensation is reached with
55 grains of moisture per pound of dry air, a
condition which would be met at any of the following
data center control settings:
• 60% RH @ 65° F
• 50% RH @ 70° F
• 45% RH @ 75° F (Quite common)
• 36% RH @ 80° F
Specifying Cooling Unit Set Point
6/16/2016 © Upsite Technologies, Inc. 28

29. SPECIFYING CAGES/LAYOUT THAT IS
COMPATIBLE WITH CONTAINMENT
© Upsite Technologies, Inc.6/16/2016 29

30. Cages and Containment
• EVERYTHING!
• Cages and containment don’t have to be mutually
exclusive
• If the cage is not compatible with containment, there
will be a need for extra volume airflow and lower
cooling unit set points, resulting in higher operating
costs
Specifying Cages Compatible with
Containment Affects:
6/16/2016 © Upsite Technologies, Inc. 30

31. • Chimney cabinets provide full containment can be
deployed independent of layout
• Out of a total 79 cabinets, 12 problem cabinets fitted
with chimney exhaust
• 38% chiller plant energy savings
• 33.5% CRAH fan energy savings
Cages and Containment
6/16/2016 © Upsite Technologies, Inc. 31

32. AIRFLOW MANAGEMENT BEST
PRACTICES
© Upsite Technologies, Inc.6/16/2016 32

33. • EVERYTHING!
• Reduce required airflow rate
• Reduced fan energy cost
• Increase cooling unit set points / supply air temperature
• Increase cooling capacity
• increasing free cooling hours
• Lowering chiller energy costs
• Defer capital expenditure
• Cooling units or even new data center
Airflow Management Best Practices
Adhering to Airflow Management Best
Practices Affect:
6/16/2016 © Upsite Technologies, Inc. 33

34. • Responsibility of IT
• Commonly cabling impedes conditioned airflow
under the raised floor
• Commonly cabling impedes exhaust flow out of IT
equipment
Airflow Management Best Practices
Cable Management
6/16/2016 © Upsite Technologies, Inc. 34

35. • Seal cable openings
• Seal under power distribution units (PDU) and other
equipment
• Check perimeter of raised floor plenum
• No misplaced perforated
Airflow Management Best Practices
Raised Floor Open Area Management
6/16/2016 © Upsite Technologies, Inc. 35

36. Airflow Management Best Practices
Raised Floor Open Area Management
Raised floor openings sealed with brush grommets
6/16/2016 © Upsite Technologies, Inc. 36

37. • Blanking panels
• Rail seals
• Under racks
• Few sites have done this
fully
• Cabinet design plays a
large role in AFM
• Cabinet AFM design
needs to be considered
by IT when purchasing
cabinets
Airflow Management Best Practices
Rack Open Area Management
6/16/2016 © Upsite Technologies, Inc. 37

38. • Gaps between racks
• Missing racks
• Doors on the ends of aisles
• Baffles or full roof over racks
Airflow Management Best Practices
Row Airflow Management
6/16/2016 © Upsite Technologies, Inc. 38

39. Angled Rack Top Baffles (Cold Aisle)
Adjustable Rack Gap Panel
Vertical Rack Top Baffles (Hot Aisle)
Modular Containment
6/16/2016 © Upsite Technologies, Inc. 39

40.  A holistic approach
 An iterative process
 ‘Check in’ at the room level
after making any AFM
improvements
Watch video at:
upsite.com/resources/airflow-management/
Upsite’s 4 R’s of Airflow Management™
6/16/2016 © Upsite Technologies, Inc. 40

41.  June 22: The 4 Delta T’s of Airflow Management
Presented by Lars Strong, P.E.
 Register at upsite.com/airflow-management-awareness-
month
Upcoming Schedule
© Upsite Technologies, Inc.6/16/2016 41

42. Thank you!
@UpsiteTech
blog.upsite.com
On LinkedIn
Lars Strong, P.E.
Senior Engineer, Upsite
Technologies
lstrong@upsite.com
Follow Upsite for the latest
news and information on
data center AFM.
Ian Seaton
ianseaton62@hotmail.com