Data Center of the Future: Equinix Test-Drives New Power, Cooling Solutions

The data center of the future is coming, and some of the technologies that may shape it are coming together in a unique facility in Ashburn, Virginia.

Digital infrastructure specialist Equinix has created a Co-Innovation Facility (CIF) at the DC15 data center, where it operates liquid-cooled servers supported by fuel cells, sodium-ion batteries and intelligent power management. The facility is part of Equinix’s Data Center of the Future initiative, which enables partners to test-drive new equipment and software in Equinix facilities.

The Ashburn testbed features technology from Bloom Energy, liquid cooling company ZutaCore, software-defined power specialist Virtual Power Systems (VPS), and battery maker Natron Energy. The facility came online in November, and will provide data on performance and cost that may lay the groundwork for future deployment in Equinix data centers.

“The data center of the future must be sustainable,” said Raouf Abdel, EVP, Global Operations at Equinix, who said the CIF would help the company “make significant advancements in the way we design, build and operate our global platform, with high energy efficiency standards.”

Bringing Sustainability Gains to a Wider Audience

These innovations are typically seen in hyperscale operators rather than service providers such as Equinix, which provides colocation services for thousands of customers with a wide range of IT equipment. With nearly 250 data centers, Equinix has the ability to sustainability and efficiency improvements to its 10,000 customers.

The CIF is a response to real-world challenges being felt across the industry, said David Hall, Fellow for Technology and Architecture in the Office of the CTO at Equinix.

These include the growing urgency of climate change, power constraints in leading data center markets, and increases in rack density as customers deploy more powerful hardware.

The Co-Innovation Facility in Ashburn features 400 CPUs, with 200 CPUs using liquid cooling and the other 200 using air cooling. This allows Equinix to benchmark their performance against traditional designs.

Bloom Energy and UPS-Less Data Centers

The CIF is using onsite solid oxide fuel cells from Bloom Energy to generate redundant energy, an approach that can be used to eliminate the need for fossil-fuel powered generators and power consuming Uninterrupted Power Supply (UPS) systems.

Bloom Energy Server fuel cells at the Equinix SV 5 data center in San Jose. (Photo: Rich Miller)

Fuel cells are not new to Equinix, which has deployed more than 50 megawatts of Bloom Energy gear powered by natural gas in high-cost energy markets like California, New York and Massachusetts. In its newest San Jose data center, Equinix uses the Bloom systems as primary generation with utility electrical grid and generators as backup sources.

Hall says this may become a more common approach in power-constrained markets around the world, where fuel may be more available than electricity.

“We increasingly find that the grid is a limiting factor in places where we want to build new data centers,” said Hall. “Using fuel cells in island mode is something we see as being more important in the future.”

Fuel cells could also be important in edge computing deployments, perhaps using tanks of liquified natural gas to support them through outages, Hall said.

“By generating power onsite with high levels of power availability and needed resiliency, data centers can take control of their sustainable energy needs,” said Deia Bayoumi, Vice President, Product Management at Bloom Energy. “Bloom’s technology eliminates reliance on electricity utilities to meet capacity requirements as well as highly pollutive diesel generators to provide backup power.”

Liquid Cooling With ZutaCore

Like most data centers, Equinix uses air to cool the servers in its data centers, where it leases space to customers. Liquid cooling is used primarily in technical computing, enabling the use of more powerful hardware that uses more energy. Equinix has been testing the use of liquid cooling in its data centers, and hopes to use the technology in its Equinix Metal service to create a high-density, energy efficient computing platform.

At the CIF, Equinix is running 200 CPUs using a direct-on-chip, waterless, two-phase liquid cooled rack systems. ZutaCore uses Novec, a dielectric liquid coolant from 3M that has proven popular in immersion cooling. Instead of dunking servers in a bath, ZutaCore is delivering the dielectric fluid directly to the processor, using a piping system to bring the liquid inside the server chassis. ZutaCore uses two-phase cooling, in which heat from the chip is dissipated by interacting with a fluid, which is vaporized by the heat.

These racks are capable of cooling upwards of 100 kW per rack in a compact design. Equinix believes this approach can dramatically shrink its data center footprint, reducing its use of energy, land, construction materials and water.

Liquid-cooled servers represent a small percentage of capacity today, but that will change quickly in coming years, Hall said.

“Our traditional IBX (colocation facility) is getting denser,” said Hall. “We see an average of something like 8 kW a rack. Our estimate is that this average is going to increase significantly. For an Equinix Metal deployment, this may be something like 19 kW in the short term, but by 2025 it may be 50 to 60 kW.

“I think that there’s an opportunity for a next generation of Equinix IBXes where we anticipate maybe three quarters of the load will be liquid-cooled.” said Hall.

“I think that there’s an opportunity for a next generation of Equinix IBXes where we anticipate maybe three quarters of the load will be liquid-cooled.”
David Hall, Equinix

ZutaCore President Udi Paret says the Equinix partnership can “advance the proliferation of liquid cooling on a global scale.”

“Together we aim to prove that liquid cooling is an essential technology in realizing fundamental business objectives for data centers of today and into the future,” said Paret.

Software-Defined Power With Sodium-Ion Batteries

One of the most interesting new technologies at the CIF is the use of software-defined power technology from VPS, combined with cabinet-mounted energy storage from Natron Energy using sodium-ion batteries. The combination of cabinet-level power management and battery energy storage has the potential to create a 30 to 50 percent improvement in power efficiency, Equinix said.

VPS uses a combination of software and distributed batteries to conduct peak shaving, using the batteries to store power and allocate it to the system when needed, creating a more elastic system for distributing power. That includes reallocating power from underused racks to IT gear that needs additional power.  The batteries and power flows are managed by the company’s software suite, which allocates power from by utility sources, UPS systems and batteries.

VPS has worked primarily with lithium-ion batteries, but the company has also recently begun working with sodium-ion technology from Natron Energy, which offers the potential to pack more capacity in a smaller footprint, as well as faster recharging.

“Data centers of the future are software-defined,” said Dean Nelson, CEO of Virtual Power Systems. “Our collaboration with Equinix, Natron Energy and CE+T in the Co-Innovation Facility showcases how VPS software unlocks stranded power driving up utilization and enabling cloud-like flexibility on-prem.”

Hall said intelligent power management could be particularly important when combined with fuel cells. “We think that in a scenario where you deploy fuel cells, you don’t want them specified to your peak load. In this scenario, we anticipate we see some sort of use of stored energy in batteries.

“We understand lithium-ion batteries very well, but we are interested in other types of battery technologies,” Hall added. “This is an opportunity to test-drive these. We think there’s some loads where we would have to charge and recharge often. The chemistry for non-lithium batteries. is interesting for this use.”

Future Path for Innovation

Equinix will monitor the technologies at the CIF with an eye toward deploying them in future data center builds. That will likely happen first at Equinix Metal, a cloud-style service where Equinix  leases capacity on bare metal servers, which can be provisioned through a web interface. Equinix manages the physical environment, including the cooling. As a result, Metal offers a platform for innovations which can then be offered to colocation customers who control their own hardware, with Equinix providing space in cabinets and cages.

“Metal is a platform that we can use for testing,” said Hall. “Metal has great orchestration and monitoring tools that we can use.”

Hall believes these new approaches will not just be more sustainable, but also have the potential to reduce the cost of IT services without impacting reliability.

“Equinix has made significant commitments to sustainability,” said Hall. “It’s important to us as a company, and our customers are asking us to do more. But the other piece of this is that customers want us to be able to do that in a reasonable cost and in a manner which delivers high levels of availability.

“One of the greatest challenges in colo is that you essentially over provision everything which is done at the expense of efficiency. One thing we want to do is give customers the ability to run their IT more efficiently. I think it’s great for customers, great for Equinix and it’s also great for the planet.”