Mikros Technologies and Carbice Corporation are adapting technology originally created for the International Space Station to solve heat-dissipation problems in AI data centers. Mikros CEO Drew Matter and Carbice CEO Baratunde Cola told EE Times in exclusive interviews that increasing chip heat-emission issues in AI data centers are driving demand for liquid cooling and new thermal interface materials (TIMs), such as the carbon nanotube parts made by Carbice.
The CEOs expect their components to help prevent the “throttling” caused by overheating that forced the May 7 shutdown of an Amazon data center in Virginia. The outage impacted companies, including cryptocurrency exchange Coinbase, according to Reuters.
Data center operators typically use throttling to manage heat, power consumption, or traffic, yet it carries significant costs, including long-term performance and reputation issues. Emergency thermal throttling when servers overheat can result in downtime costs of up to $540,000 per hour, according to U.S. engineering services provider Ketchum & Walton.
Data centers’ electricity demand in 2030 will increase to nearly 12% of total U.S. annual consumption, according to the Environmental and Energy Study Institute, a Washington, D.C.–based nonprofit organization. Data center electricity consumption is stressing power grids in the U.S.
Broadcom has partnered with Mikros for its 3.5D eXtreme Dimension SiP, a stacked-die platform that combines 2.5D and 3D integration, enabling AI customers to develop custom accelerators (XPUs) and compute ASICs.
Drew Matter_Mikros CEO
Mikros Technologies’ Drew Matter
“Broadcom is designing the next-gen XPUs that are going to help the inference community over the next few years, and they look to us to be a driving thermal partner to dissipate the heat on these incredibly high-powered chips,” Matter told EE Times.
Mikros, acquired by Jabil in 2024, did designs for the International Space Station 30 years ago to cool high-power-density electronics. The company is adopting those designs for high-performance computing and AI.
“By partnering with Mikros and Jabil, we are ensuring our custom 5-kW XPU customers have access to a robust ecosystem of high-performance cooling,” Ken Kutzler, VP of AI systems development at Broadcom, said in an April announcement. “Their microchannel technology provides the critical chip-level thermal resistance necessary to unlock the full performance of our ASICs, enabling a seamless chip-to-chiller liquid-cooling path essential for the next generation of high-density AI data centers.”
Liquid cooling
Upcoming data centers are adopting liquid cooling instead of less efficient air cooling, Matter said. Liquid cooling can help cut more than a third of the energy use in air-cooled data centers that is spent running fans and A/C equipment rather than compute.
“Chips are coming out in the next three years, and they’re being designed for liquid,” Matter said. “Liquid cooling is going to be fully the norm in probably three to five years. We are looking at savings with liquid cooling that could easily be upwards of a million dollars a rack over the lifetime of the rack.”
The company’s MikroMatrix design allows its cold plates to distribute coolant to match the power maps of next-generation GPUs.
“It also allows us to have very even cooling over the chip, and because we have this ‘pixel’ design,” Matter said. “When you have a new GPU with multiple zones, we can actually tailor the pixels to match the zones, and that provides for even more efficiency.”
Marvell Technology
Mikros is also working with Marvell Technology on custom designs for co-packaged optics chips. “They’re actually bringing the optics in toward the chip, and so it was actually a cold plate that cooled not only the chip itself but the optics around it in one solid cold plate,” Matter said. “Those transceivers in themselves don’t transmit the amount of heat that a CPU or GPU would, but when packaged all together and close to the GPU, they can actually be affected by the GPU’s heat, so you need to be cooling the switch chip.”
Transitioning to liquid cooling enables higher densities of racks in an AI data center, Marvell said of the Mikros technology in a blog post last year.
The increased rack density is driven by the MikroMatrix platform using a matrix array of microchannels oriented perpendicular to the chip surface, dramatically increasing the contact area within the cold plate for better heat dissipation, Marvell noted.
“When a client works with a Broadcom or a Marvell chip, for instance, we are able to work with that client for any customization that’s needed as well,” Matter added.
Carbice
Carbice’s Cola has been shipping the company’s carbon nanotube products to satellite makers for about seven years. “Almost every single prime satellite company is a customer to some degree of our material,” he told EE Times. “When we got into compute, the best place to start was gamers.”
Last year, Carbice started supplying its carbon nanotube TIM to gaming system maker CyberPowerPC.
“We’ll ship on over half a million CPUs this year,” Cola said.
The vertical structure of Carbice nanotubes helps to dissipate heat by increasing surface area for free convection by a factor greater than 10,000, according to Cola. What differentiates carbon nanotubes from other TIMs is the ability to keep the cold plate stuck to the chip, he added.
Baratunde Cola_Carbice
Carbice’s Baratunde Cola
“It’s a mechanical solution,” Cola said. “Carbice looks like an array of curly fries standing up. Those curly fries can both compress and extend. Every chip that’s made changes curvature with temperature, because the chips are multi-material. They have thermal-expansion mismatch built into the manufacturing. You have a single point of failure at the connection point of the cold plate to the chip. Just because of the viscosity and the stress profile—and you cannot escape it—there will be random throttling events, random failures.”
With heterogeneous integration, material stacks are constantly changing, and the curvature problem becomes even more pronounced, Cola noted.
“Our stuff rides the change in curvature without losing contact,” he said. “The product we make is a simple sticky note. It’s just a peel-and-stick product.”
Carbice is expanding its production facility in Atlanta near the Georgia Institute of Technology, where Cola is a professor of thermal management. The facility will triple Carbice’s output from a year ago to supply contract manufacturing OEMs such as Jabil and Flex, according to Cola.
“Our stuff’s been running in the Georgia Tech data center,” he said. “Georgia Tech gets early Nvidia chips—there’s a partnership. We’ve been running on Nvidia chips in the Georgia Tech data center for three years, so we know the failure rates.”
TIMs range from greases, gap pads, and phase-change materials to nanotubes and graphite solutions.
“When we interact with a client like Broadcom, for instance, we’re going to be looking for a TIM that best matches that need,” Matter said. “Sometimes, we’ll have to choose from one family or another based on size or performance or tolerances.”
The carbon nanotube TIM from Carbice has strengths in its ability to adjust to chip changes over time, Matter said.
“We know that team very well,” he added. “They have a good product, and they’re very useful in some applications. There are various applications where customers will ask us for a specific type of TIM. Sometimes it might be carbon-nanotube-focused; others might have different desires for the way that their thermal interface works, depending on their system.”