Sharon AI Launches Cloud Media Tech with On-Demand GPU Service
NVIDIA H200
Tensor Core GPU
The NVIDIA H200 Tensor Core GPU, based on the NVIDIA Hopper™ architecture, is engineered to accelerate generative AI and high-performance computing (HPC) tasks. Featuring 141GB of HBM3e memory and advanced processing power, it doubles the performance of previous models like the H100, making it the ideal choice for AI factories, large-scale inference models, and scientific computing.
Basic Product Information
Product Name
NVIDIA H200 Tensor Core GPU
Architecture
NVIDIA Hopper™
Memory
141GB HBM3e
Compute Power
Up to 4 PetaFLOPS of FP8 performance
Release Year
2024
Use Cases
AI inference, large language models (LLMs), scientific computing, HPC workloads
Key Advantages
141GB HBM3e Memory
Offers larger and faster memory for high-performance tasks.
4.8TB/s Memory Bandwidth
Nearly 1.4X more than the H100 GPU for faster data processing.
Up to 4 PetaFLOPS
Industry-leading FP8 performance.
2X LLM Inference Performance
Perfect for large language models like Llama2.
Energy Efficiency
Greater performance at the same power profile as the H100.
Specifications
Performance Specifications
FP8 Performance
4 petaFLOPS
LLM Inference Performance
2X compared to H100
HPC Performance
110X faster time to results
Memory Bandwidth
4.8 TB/s
FP64
34 TFLOPS
FP64 Tensor Core
67 TFLOPS
FP32
67 TFLOPS
TF32 Tensor Core
989 TFLOPS (with sparsity)
BFLOAT16 Tensor Core
1,979 TFLOPS (with sparsity)
FP16 Tensor Core
1,979 TFLOPS (with sparsity)
INT8 Tensor Core
3,958 TFLOPS (with sparsity)
Decoders
7 NVDEC, 7 JPEG
Confidential Computing
Supported
Multi-Instance GPUs
Up to 7 MIGs @18GB each
Memory and Bandwidth
GPU Memory
141GB HBM3e
Memory Bandwidth
4.8TB/s
Thermal and Power
Max Thermal Design Power (TDP)
Configurable up to 700W
Cooling
Active and passive cooling options available
Board Specifications
Form Factor
SXM or PCIe (depending on the model - H200 SXM or H200 NVL)
Interconnect
NVIDIA NVLink: 900GB/s PCIe Gen5: 128GB/s (for H200 NVL) 2- or 4-way NVIDIA NVLink bridge: 900GB/s (for H200 SXM)
Supported Technologies
Multi-Instance GPU (MIG)
Up to 7 MIGs per GPU (18GB each)
Confidential Computing
Fully supported for secure AI processing
AI Enterprise Software
NVIDIA AI Enterprise included for streamlined deployment of generative AI solutions
Server Compatibility
Compatible with
NVIDIA HGX™ H200, NVIDIA MGX™ H200 NVL, and NVIDIA-Certified Systems™ with up to 8 GPUs.
Additional Features
01 Efficient for Large Language Models
Handles models like GPT-3 with ease, providing 2X throughput compared to H100 GPUs.
02 Enterprise-Ready
Includes NVIDIA AI Enterprise software, which offers stability, security, and accelerated AI deployment.
