Blackwell GPU Architecture

In recent years, the pace of artificial intelligence development has been astonishing. The explosion of generative AI, especially the emergence of large language models, has allowed people to truly experience the power of AI for the first time. Models such as ChatGPT, Claude, and Gemini can not only write articles and generate code but also perform complex reasoning and dialogue. Yet behind these capabilities lies a major challenge: the compute bottleneck. The larger the model and the more data it consumes, the more exponentially its demand for computing resources grows. Traditional GPU architectures are increasingly struggling to keep up with these demands.

This is why NVIDIA introduced the Blackwell GPU architecture. The Blackwell GPU is not only designed to solve today’s compute challenges but also to lay the foundation for the AI Factory era of the next decade. The concept of an “AI Factory” is not a metaphor but an accurate description of the future of AI infrastructure. Enterprises and research institutions will no longer train a single model in isolation; instead, they will run multiple models simultaneously, like an assembly line, carrying out large-scale inference, training, and optimization. The Blackwell GPU was created precisely to meet this demand.

Blackwell’s design represents a fundamental breakthrough. Its most striking feature is the dual-chip design, known as the Blackwell Ultra architecture. Take the B200 as an example: it is essentially composed of two GPU dies, tightly coupled through NV-HBI high-speed interconnect technology. With a bandwidth of up to 10 TB/s, NV-HBI enables nearly bottleneck-free data exchange between the two chips, far surpassing traditional PCIe or NVLink connections. This design allows Blackwell to maintain the form factor of a single card while delivering double the compute power and higher energy efficiency.

At the same time, Blackwell introduces fifth-generation Tensor Cores. These cores are the “engines” of deep learning computation, designed to accelerate matrix operations. Compared to Hopper’s fourth-generation Tensor Cores, Blackwell’s new cores are significantly more efficient at handling low-precision calculations. In FP8 and NVFP4 precision modes, throughput has increased by an order of magnitude, providing a solid hardware foundation for large-scale model training and inference.

For system expansion, Blackwell also brings the NVLink Switch System. This system allows thousands of GPUs to be interconnected into a massive compute cluster. With interconnect bandwidth reaching 1.8 TB/s, data transfer between GPUs is virtually latency-free. This capability is critical for AI factories, where large-scale model training requires frequent parameter and gradient exchanges across thousands of GPUs. Without efficient interconnects, even the most powerful compute resources would be hampered by communication bottlenecks.

Beyond architectural breakthroughs, Blackwell also innovates in precision formats. Traditional AI inference tasks often use FP8 or BF16 precision, which balance accuracy with energy efficiency. However, NVIDIA has introduced a brand-new NVFP4 precision format in Blackwell. This low-precision format is specifically optimized for inference, reducing computational and storage overhead while maintaining model accuracy.

According to official data, when using NVFP4 for inference, Blackwell achieves up to 25 times higher energy efficiency and up to 30 times faster inference speed. This means that under the same power consumption, Blackwell can handle far more inference tasks, making it especially suitable for high-concurrency online services. Whether it’s language models like GPT or multimodal models such as Claude and Gemini, Blackwell enables faster and more cost-effective inference.

This innovation in precision is not just a technical optimization but also a deep understanding of AI application scenarios. As large models move toward commercialization, inference costs have become one of the top concerns for enterprises. NVFP4 directly reduces power consumption and latency, making the deployment of large models far more feasible.

If single-GPU performance improvements are Blackwell’s first step, then system-level integration and scalability are its second. Blackwell does not exist in isolation; it is typically paired with the Grace CPU to form the GB200 superchip system. This combination not only improves memory access efficiency but also enables tighter coordination in task scheduling and data transfer. The Grace CPU provides powerful general-purpose compute, while the Blackwell GPU focuses on AI acceleration. Together, they form an end-to-end compute platform.

More importantly, Blackwell’s design philosophy is oriented toward AI factories. It supports clusters of thousands of GPUs, combined with NVLink, liquid cooling, and rack integration solutions, to build true AI factory-grade infrastructure. This system-level solution is not just about stacking hardware but about optimizing hardware and software together. On the software side, NVIDIA provides CUDA, TensorRT, and other tools to help developers fully unleash Blackwell’s potential.

If the Hopper H100 was once the benchmark for AI compute, then Blackwell represents a generational leap forward. The most obvious change is in compute performance. Hopper’s FP8 performance is around 4 PFLOPS, while the Blackwell B200 reaches 20 PFLOPS—five times higher. With the addition of fifth-generation Tensor Cores, inference speed can improve by up to 30 times, a difference that is hard to ignore.

In terms of chip scale and memory configuration, Blackwell also shows overwhelming advantages. Hopper has about 80 billion transistors, while Blackwell surpasses 208 billion—nearly three times more. For memory, Hopper uses HBM3, while Blackwell upgrades to HBM3e with a capacity of up to 192 GB. Such large memory capacity is critical for training massive models, as it reduces distributed training overhead and improves efficiency.

Interconnect capability is another highlight. Hopper’s NVLink bandwidth is 900 GB/s, while Blackwell, with the NVLink Switch System, doubles this to 1.8 TB/s. This improvement is crucial in large-scale clusters, where the efficiency of parameter synchronization and data exchange directly determines overall system performance.

In terms of precision formats, Blackwell once again takes the lead. Building on Hopper's optimizations for FP8, Blackwell introduces NVFP4, striking a better balance between energy efficiency and accuracy. With inference energy efficiency improved by up to 25 times, this is a transformative improvement for large-scale inference applications.

Finally, Blackwell demonstrates a different approach to system-level collaboration. Hopper was more often deployed as a standalone chip, while Blackwell emphasizes deep integration with the Grace CPU, forming the GB200 superchip system. This end-to-end solution marks a paradigm shift from “single-chip performance” to “system-level collaboration,” making Blackwell the true compute engine of the AI factory era.

These performance improvements translate into tremendous value in real-world applications. In large model training and inference, Blackwell can significantly shorten training cycles, reduce energy consumption, and improve inference speed. For tasks such as LLMs, VLMs, and RAG, Blackwell provides unprecedented compute support.

In biotechnology, Blackwell’s potential is equally significant. Tasks such as protein structure prediction and genome analysis demand enormous compute resources. Blackwell’s high throughput and large memory capacity can dramatically improve efficiency in these areas. For drug discovery and precision medicine, this means faster experimental cycles and lower costs.

In fields such as financial risk control, autonomous driving, and robotics, Blackwell’s low latency and high energy efficiency also provide strong support. Financial institutions can perform risk modeling and real-time monitoring more quickly, autonomous driving systems can process sensor data more efficiently, and robots can make smarter decisions in complex environments.

Looking ahead, Blackwell is more than just a GPU—it represents a paradigm shift in compute. From “GPU chips” to “AI factories,” this is a strategic upgrade. NVIDIA is no longer just providing individual chips but delivering complete system-level solutions. The impact on AI infrastructure and cloud service providers is profound. Future competition will no longer be about single-chip performance but about system-level compute. Whoever can provide more efficient, flexible, and scalable AI factories will hold the advantage in this race.

The emergence of the Blackwell GPU is not merely a hardware upgrade but a forward-looking strategic move. Through its dual-chip design, NVLink Switch System, NVFP4 precision, and deep collaboration with the Grace CPU, it builds a complete system-level compute ecosystem. These innovations make the Blackwell GPU not just a chip but the core engine of the AI factory.

In terms of performance, the Blackwell GPU achieves leaps across nearly every dimension: stronger compute power, larger memory, faster interconnects, and higher energy efficiency. These improvements translate directly into real-world value, enabling faster training, more efficient inference, and more feasible deployment of large models.

Yet the Blackwell GPU is not the endpoint of compute—it is a new beginning. As AI models continue to grow in scale, future compute demands will only increase. Blackwell points the way forward: compute must not only be stronger but also more efficient, more flexible, and more system-oriented. It stands as a milestone on the road to the AI factory era and the starting point of the next compute paradigm.