The NVIDIA H100 NVL, built on the NVIDIA Hopper™ architecture, delivers industry-leading performance, scalability, and security for AI and high-performance computing (HPC) workloads. With innovations such as the Transformer Engine and fourth-generation Tensor Cores, it accelerates large language models (LLMs), securely handles enterprise workloads, and supercharges AI training with up to 4X performance improvement over previous models.
Up to 5X over NVIDIA A100 systems for LLMs up to 70 billion parameters
FP64
30 teraFLOPS
FP64 Tensor Core
60 teraFLOPS
FP32
60 teraFLOPS
TF32 Tensor Core
835 teraFLOPS (with sparsity)
BFLOAT16 Tensor Core
1,671 teraFLOPS (with sparsity)
FP16 Tensor Core
1,671 teraFLOPS (with sparsity)
FP8 Tensor Core
3,341 teraFLOPS (with sparsity)
INT8 Tensor Core
3,341 TOPS (with sparsity)
Decoders
7 NVDEC, 7 JPEG
Confidential Computing
Supported
Multi-Instance GPUs
Up to 7 MIGs @12GB each
Memory and Bandwidth
GPU Memory
94GB HBM3
Memory Bandwidth
3.9TB/s
Memory Clock
2619 MHz
Thermal and Power
Power
Configurable between 350-400W
Server Options
Compatible with partner and NVIDIA-Certified Systems that can accommodate 1 to 8 GPUs
Board Specifications
Form Factor
PCIe dual-slot air-cooled
Interconnect
NVIDIA NVLink: Offers 600GB/s bidirectional bandwidth
PCIe Gen5: Supports up to 128GB/s
NVLink Bridge: Can be connected with another H100 NVL using 2- or 4-way NVLink bridges for increased bandwidth
Supported Technologies
Multi-Instance GPU (MIG)
Up to 7 MIGs @ 12GB each
Confidential Computing
Provides hardware-based security for data in use.
AI Enterprise Software
NVIDIA AI Enterprise included for secure and scalable AI deployment.
Server Compatibility
NVL
NVL: Compatible with Partner and NVIDIA-Certified Systems with 1-8 GPUs
NVIDIA AI Enterprise: Included subscription