Huawei AI CloudMatrix is power-hungry over Nvidia GB200, but that’s fine
Huawei AI CloudMatrix tech has occupied a huge space on the internet for being a good yet power-hungry alternative to Nvidia GB200 NVL72. But this particular point doesn’t seem to be a major drawback, especially for Chinese customers.
SemiAnalysis recently noted that Huawei AI CloudMatrix uses 4x more power than Nvidia GB200. It is so because the company highly depends on brute force.
Brute Force refers to a process where more processors are installed on a board to achieve better performance for AI. Since Huawei can’t access top-end technologies to manufacture advanced chips for AI, it uses the Brute technology.
Although that often leads to more power consumption. SemiAnalysis said that Huawei’s new multifaceted plan in the AI field includes a dual-chiplet Ascend 910C chip, optical interconnections, and AI CloudMatrix relying on proprietary software.
This whole mechanism delivers a 2.3x lower performance per watt than Nvidia’s GB200 NVL72. But that will still allow Chinese firms to efficiently train advanced AI models. How? Let’s find out!
CloudMatrix AI chip tech consists of 384 Ascend 910C chipsets placed in an all-to-all mesh network (fully optical). It is spread across 16 racks (twelve computing racks with 32 accelerators each and four networking racks with high bandwidth).
Instead of copper wires, Huawei AI CloudMatrix uses optics for intra- and inter-rack connectivity, resulting in high cluster communication bandwidth.
Speaking of performance, the CloudMatrix 384 offers 300 FLOPs of dense BF16 computer (two times of Nvidia GB200 NVL72), 2.1 times more total memory, 2.1 times high scale-up bandwidth, 5.3 times scale out bandwidth, and 3.6 times greater HBM capacity over Nvidia’s technology.
But it’s 2.3 times less power-efficient per FLOP, 1.8 times less efficient per TB/s of memory bandwidth, and 1.11 times less efficient per TB of HBM than Nvidia.
That increases the entire system’s power requirement by 559 kW, which is only 145 kW for GB200. But that’s not a major drawback, as the average electricity cost in mainland China is $56 MWh. Thus, Huawei approach to AI isn’t bad.
Moreover, if the company can deliver its CloudMatricx 384 in abundance with accurate software support, then power consumption won’t be a big deal for its customers.
Huawei AI CloudMatrix is power-hungry over Nvidia GB200, but that’s fine (Image Credits: Huawei/X)
(source)
The post Huawei AI CloudMatrix is power-hungry over Nvidia GB200, but that’s fine appeared first on Huawei Central.
