Liquid Cooling Solutions for DATA CENTERS - R.M.IYENGAR BLUESTAR LIMITED.

Transcription

1 Liquid Cooling Solutions for DATA CENTERS - R.M.IYENGAR BLUESTAR LIMITED.

2 Presentation Goals & Outline Power Density Where we have been- where we are now - where we are going Limitations of Air Cooling Air Distribution, Effectiveness and Heat Removal What Does ASHRAE Say? Unified System Solutions Energy Efficiency

3 Common Data Center Scene Have you seen data centers that look like this?

4 Thermo photo Analysis

5 IT Equipment Trends This area is for the equipment footprint only! Power density is diluted when the entire data center area is added. *from The Uptime Institute (

6 Issues in the Data Center The rise of heat densities at the microprocessor level (watts)

7 Are We Hitting the Wall? The failure rate at the top third of the rack is three times greater than at the bottom. Mr. Kenneth Brill, Executive Director: The Uptime Institute, March 2003 DataCenter Dynamics Conference) The wall is at 3.5 KW/Cabinet (with raised floor) Without raised floor, 2.5 KW/Cabinet max. Steve Spinazzola, RTKL, Oct. 2004, 7x24

8 How Do We Remove the Heat? Water Water has 3500 times more heat capacity than air by volume Air Overhead Under floor

9 Overhead Air Distribution Limitations to capacity due to duct size / space constraints Higher cost for ductwork versus underfloor system

10 Underfloor Air Distribution Increased floor depth for higher flows Limitation of perforated tiles (400 cfm = +- 4 kw/rack) UF obstructions Air Misdistribution underfloor velocity/static pressure Air plumes

11 The Cause of Flow Misdistribution

12 Above floor Plumes Rack total airflow: 350cfm Max inlet temp: 16C/61F Rack total airflow: 700cfm Max inlet temp: 18C/64F Rack total airflow: 1400cfm Max inlet temp: 28C/82F

13 Underfloor Plumes Row Velocity vectors & pressure (inch wg ) under the raised floor Column 1 kw / Cabinet 24 Raised Floor

14 HVAC Effectiveness Bypass Airflow Air Distribution Efficiency 70% of cooling supply air is wasted and bypasses back to return system.

15 ASHRAE Standards Key Points of ASHRAE TC9.9 Equipment Environmental Specifications Equipment Room Airflow Standardize Measuring / Monitoring Points Equipment Airflow Protocol Syntax Equipment Manufacturer s Heat Reporting

16 Equipment Room Airflow ASHRAE Standards

17 The New Paradigm Extraction Versus Dilution

18 AIR COOLED EXTRACTION SYSTEM

19 POINTS TO CONSIDER Effective for standard racks up to 6 KW only No cabinet doors. Cable management behind servers in hot aisle. Limit air bypass use rack blanking panels and seal all floor cutouts. Limit mixing between hot and cold aisles. Partition panels above racks and doors at aisle ends. Sprinklers / FP piping in each aisle.

20 Rack Cooling Unit Options Down flow Units Centrifugal fan drive EC fan drive With DX system With chilled water

21 High Density Cooling Solution

22 Heat Density Cabinet Pre P3 Processor P3 Mobile Blade P4 P4 Blades/P4 Multi- Single Processor Processor 1U & 2U NextGen Ultra Dense Blade & 1U Server High-End Enterprise Servers

23 Cooling Hierarchy Cooled room, open racks Plenum distribution, server racks Forced air racks Water cooled racks Water cooled CPU 1 kw 1-3kW 3-6kW 6-20kW 20-35kW 35kW+

24 Background to the Need for Water Cooling Rising heat densities Limitations of air cooling Focus on heat removal at source Part load operating conditions Energy efficiency

25 COOLING SYSTEMS CLOSER TO THE LOAD. Mounting the cooling modules as close as possible to the source of heat such as placing them directly above, alongside, or within high-density racks reduces the distance that the fans must move air. This can provide up to 70% savings of the energy required to move the air.

26 OPEN COOLING ARCHITECTURE. The room s air volume is then used as thermal storage to ride through short power outages. Cool room air enters the front of the server rack cabinet and absorbs heat as it passes over the electronics. It then exits the cabinet, moves back into the room, and recirculated through CRAC.

27 Closed cooling architecture Closed architecture systems involve extraction of room air in server rack cabinets that are completely sealed from room air and environmental conditions. In a closed design, air is circulated through the electronics and passes through a liquid-toair heat exchanger located in the cabinet and back to them at room temperature.

28 Rack Cooling Unit Options CDU Cooling Distribution Unit Rear Door Heat Exchangers Chilled Water Cooling

29 CDU with Rear Door Heat exchanger Heat removal at source, rejected to water not room Rear door heat exchangers on back of high heat load cabinets without taking up any floor space High density racks can be added without need to upgrade CRAC units Easily retrofitted as high density racks added Eliminates low air temp setpoints to compensate for high density racks Only relatively small chilled water plant required CDU limits volume of chilled water in computer room Reduces number / capacity of CRAC Units required Reduces floor space required for CRAC Units

30 Data Racks & Rear Door Heat Exchangers Industry Standard Data Racks 600 wide 20kW Cooling 800 wide 30kW Cooling 6 Hot Swap Fans Retro-fittable to other Racks Without swap fans for >6 kw upto 10 kw

31 Typical Installations

32 System Concept Rear Door Heat Exchangers Cooling Distribution Cabinet kW External chiller Up to 6 racks (or 12 racks at lower loads)

33 Minimal Thermal Impact Server Implementation using CDU & Rear Door Heat Exchangers C C C C C C 20 kw 20 kw 20 kw 20 kw 20 kw 20 kw CDU C C C C C C

34

35

36 CDU Cooling Distribution Unit CDU120/150 Main Components 120 to 150kW (409, ,815 BTU/Hr) capacity. Serves 1 to 6 rear door heat exchangers (RDHx) v 50/60Hz global operation (+220v option). Run/Standby pumps. Run/Standby control valves. Energy saving pump speed control. 240L/M (63.4 GPM) flow capacity (40L/M per RDHx). Dew point control. Inbuilt manifold system with flow balancing. Quick release, self sealing connections for data rack hoses. Easy fill system. UL approved

37 Hose Connections Process (to RDHx) Secondary Circuit Connection: Quick release couplings for clean efficient installation Fool-proof input and output connections Tidy hose arrangement for accessibility. Primary Chilled Water Connections (1.1/2 Hose tails) Secondary Circuit Rack Return Connections (3/4 Hose tails) Secondary Circuit Rack Flow Connections (3/4 Hose tails)

38 Controller & Display Temp (T3) & Hum Sensor Air Eliminator Rack 1 Control Valves Heat Exchanger Speed Control Rack 2 Rack 3 Rack 4 Chilled Water Flow / Return T1 PS3 PS4 # Flowmeter Bypass Valve Air Eliminator Filter Distribution manifolds Flow Control 40L/M Quick Release Couplings Rack 5 Rack 6 Filter # Redundant 2-way valve Level Sensor. T2a T2b PS1 Pump P1 PS2 40L/M 40L/M 40L/M Rack 3 Rack 4 # # Redundant pump Pump # # P2 40L/M 40L/M Rack 5 Rack 6 Fill point Priming Pump P3 DATE 18/05/07 DR N DSM TITLE CDU PIPE SCHEMATIC DRG. No. A3NAD371 Iss. A

39 Modular unit of 20kW Can serve one RDHx or up to four 5kW horizontal spot cooling cassettes. Can be installed in isolation or in purpose cabinet (pictured) for up to 120kW. Purpose built cabinet includes chilled water manifold system & power distribution module. Easy fill system Dew point control Quick release, self sealing connections for data rack hoses Modular CDU

40

41 Typical Manifolds

42

43

44 Graphical Display and Keypad set point 18.0 C Power On indicator (red) Graphical display area dw o/r act. temp.0 C 18.4 C Level 1 alarm indicator (flashing amber) Level 2 alarm indicator (flashing red) ESC button -To return to System (default) screen Arrow buttons For menu navigation and value adjustment OK button To enter or accept the displayed value or command & to acknowledge alarms

45 Connectivity via Webserver, Modbus & Echelon BMS integration to facilities and connectivity to IT via IP & SNMP.

46