This article is the fifth in a series of Data Centre Best Practice articles provided by Future-tech Ltd.
In this article we seek to highlight the basic best practices for deploying IT hardware in a data centre in order to maximise the available resources and minimise the potential for introducing problems.
The key essential element in all of these practices is to ensure close communication and liaison between the separate teams responsible for data centre operations and ongoing service continuity. This specifically relates to the facility engineering / operations teams and IT hardware management / installation teams.
IT Equipment Deployment
Facilities Management (Mechanical & Electrical Engineering) teams should participate in, and sign off on, IT equipment selections to make sure the selected equipment can be properly hosted in the site chosen and within space allocated. It is especially important that appropriate power and cooling supplies are available in the specified location.
Change control activities for IT Moves, Adds and changes should always include the FM / M&E team with their participation in the appropriate Change Advisory Board (CAB). FM / M&E teams should always participate in, and sign off, all IT equipment installations once completed.
All new deployments should include a strict adherence to capacity planning projections, corporate equipment purchasing and deployment policies and site resilience/ redundancy policies as well as being the subject of formal Change Approval. Deployment and capacity planning should be performed well ahead of any intended use of the space and be completed according to established best practice, ideally using a space planning / asset management tool. Once developed floor plans and cabinet layouts should be rigidly adhered to. Space and capacity planning ensures the best use of the available resources within the data centre by optimising equipment and cabinet layouts The use of Space Planning (and Capacity Planning), is essential to maximise the usage of the available space within the data centre to its ultimate potential.
Equipment floor plans and cabinet layouts should be completed and available in accordance with site standards and operational best practice in order to maximise the available capacity of the data centre site.
Cabinets should always be built with side panels to encourage front to back air flow with limited bypass. Empty cabinets or partially filled cabinets should always be blanked off with suitable blanking plates to prevent air bypass through empty gaps.
In non-contained aisles there should be at least 1 metre clearance between the top of the equipment cabinets and the ceiling and overhead cable trunking should not impede the return flow of hot air. Under floor cabling (both power and data), and pipe work (such as that for chilled water), should not impede under floor air flow if cold air is supplied through an under-floor plenum. Minimise all under-floor obstructions to maximise airflow if cooling air is being supplied in this way. Under-floor cabling should ideally be laid out below the intended hot aisles if cold air delivery is via an under-floor plenum in order to allow free air flow into the cold aisles. Cable trays should be wide and shallow where possible and set at consistent heights within the underfloor area.
Locate cabinets with highest equipment densities in the areas of the room most effectively supplied with cooling. The most effective cooling areas will coincide with the greatest areas of under floor static pressure. When downflow units are installed these areas are usually found in the centre of well designed data centres where opposing air flows meet in the middle of the under-floor plenum.
The areas of most effective cooling can be properly determined during the course of CFD analysis at the design stage however the use of CFD during operation post deployment is expensive and is of less value compared to cost in an operational environment. Thermal Mapping representing real time data is more beneficial during live operation
All holes or vents in the floors, walls or partitioning within the under-floor area should be sealed. Openings or vents into other areas of the building can contribute to significant amounts of leakage from the under-floor void and a consequential reduction in under floor static air pressure. This is normally a requirement of fire zone sealing. Leaks or holes should be discovered during statutory pressure testing. Any cut outs created should be sealed with grommets or approved foam to minimise under floor pressure loss.
Cuts in floor tiles should be protected with permanent plastic trim strips to prevent cables from rubbing against rough edges and should also be sealed with grommets once cables have been installed. Leakage from floor tile cuts can significantly reduce floor pressure and lead directly to inefficient cooling. If a raised floor is present and intended to be used as a supply plenum remember that is essential to maintain a constant and even Static Pressure under the raised floor to achieve effective cooling through the vented floor tiles. Consider the use of pressure sensors to balance under floor static pressure if used as a supply plenum.
Use of phases must be balanced across the UPS(s) in distribution to IT equipment within cabinets. Failure to do so could risk a phase imbalance and a potential trip of the UPS. Power phase loading at the UPS level and across the site should be monitored and kept balanced during IT equipment installation.
Assuming single phase devices are deployed within an individual cabinet all should use supplies from the same phase to prevent the potential for greater than 230 volts in the same cabinet. Only supplies from the same single phase should be provided within any individual cabinet and power supplies should never be “Daisy Chained” either within or between cabinets. It may also be that adjacent cabinets are required to be on same phase to prevent line to line voltages between cabinets. Equipment requiring 3-phase supplies within a cabinet should be clearly labelled as using 3-phase and the supplies into these cabinets should not be used to power adjacent cabinets.
To correctly protect supplies into cabinets “D” type Breakers are recommended to be used on cabinet circuits and all cabling (power and comms), should be clearly and accurately labelled.
It is worth noting as a guide that a typical single 32 Amp supply @ 230 Volts can deliver about 7kW per cabinet (Assuming Power Factor corrected equipment and N+1 redundancy if the cabinet has twin power supplies). It is also common to use a 16 Amp supply @ 230 Volts which can deliver about 3.5kW per cabinet.
IT equipment should always be purchased according to agreed standards and specifications for power, cooling and resilience etc. IT equipment selection should consider the appropriate ASHRAE environmental set points and energy efficiency expectations. Always select energy efficient IT equipment when specifying and purchasing new and replacement IT equipment. As a minimum this should be according to current Energy Star specifications. Ensure that the Delta T of the cooling system is closely matched to IT equipment specifications. This may allow a reduction in total airflow, while meeting the same cooling capacity and reducing operational costs by optimising cooling system efficiency.
Future-tech have been designing, building and managing business critical data centres since 1982. The experience gained in being involved in the data centre sector from the outset has resulted in Future-tech sites achieved 99.999% uptime during 35+ years of operation. Future-tech has a team of experienced, skilled and highly trained in-house Data Centre Engineers capable of properly maintaining and operating business critical data centre sites of all sizes. For more details please contact Richard Stacey on 0845 900 0127 or at rstacey@future-tech.co.uk