[Microsoft expected to announce the internalization of AI semiconductors, but announced the internalization of data centers]

Microsoft has developed its own semiconductor technology
It’s been announced!

• We announced the CPU as well as the AI accelerator, and the release date was early next year
• This time, we announced two types of AI accelerator called Maia 100 and Cobalt 100 CPU

(Actually, there were three types. We also announced a semiconductor dedicated to network computing called DPU.)

Recent data center semiconductors consist largely of CPU+GPU+AI Accelerator+DPU

Microsoft has virtually internalized all the lineups needed for the data center, except for GPUs

• In particular, given that data center GPUs have more AI operations than graphics, Microsoft has virtually internalized almost every major chip except graphics operations

All the newly announced chips are based on TSMC 5nm process

We’ll give you a brief comment on Telegram today and update you with more details over the weekend

Maia 100

• In conclusion, the performance of the chip itself was similar to that of the H100, and it overwhelmed other big tech chips
• Since AI semiconductors connect a large number of them in parallel, network performance is important for communication between AI semiconductors, but it surpassed even H100 in network performance
• However, the Maia 100 is expected to lag behind MI300 and H100 in actual performance with too low HBM bandwidth, contrary to expectations
• It was designed for our GPT services and due to the nature of GPT, there were many bottlenecks in the HBM sector, which was difficult to understand
• A possible hypothesis is that the Maia 100 was already designed before the GPT success
• Now that H100 has a new memory bandwidth adaptation, Microsoft is likely to do the same, or the next product will adjust the design idea
• The bottom line is that if it weren’t for the HBM bandwidth, we’d have a chip that really beats the H100. Microsoft is seen as a significant threat to the next product in the AI semiconductor competition because it is not comparable to Nvidia in terms of cost-effectiveness

Cobalt 100 CPU

• It seems to have been designed by adopting a large part of ARM’s reference design
• So it’s not as good as other companies, but it’s not expected to lag behind
• Amazon’s own CPU, Graviton, is likely to be a rival, selling outside, but actually dealing with internal cloud operations cheaper

200G DPU

• Today, DPU is a semiconductor that specializes in the computations needed for networks and storage, and aims to ease the burden on CPUs
• In fact, Microsoft has been preparing for this by acquiring DPU companies, which have completed most of the portfolio needed for the data center

Completely new rack and water cooling system

• With all the chips in place, Microsoft has completely redesigned the rack and water cooling systems that correspond to the server frame to suit its own systems

conclusion

• In fact, it internalizes all of the key compute semiconductors needed in the cloud, data center, except graphics GPUs for graphics
• Even if it wasn’t for the HBM bandwidth, the product would have been better than the H100
• It’s better to use Nvidia and AMD for the time being because of the bandwidth, but the next product will be scary
• Maso redesigns cooling systems, server structures and virtually internalizes almost all hardware like Apple
• I think it’ll be scarier in a year or two
• Portfolio to name Microsoft & Microsoft Hardware in the future