Lichtenberg II Phase I

Hardware of Phase I of Lichtenberg II

This new system is in regular operations mode since 1st December 2020.

Login (8 nodes)

  • 2x “Intel® Xeon® Platinum 9242 Processor”
  • 96 CPU cores and 2x Intel® AVX-512 units per core
  • 768 GByte Main memory (DDR4-2933)
  • 48 cores per processor – Hyperthreading is off
    • CPU clock 2,3 GHz, Turbo 3,8 GHz
    • AVX-512 (Advanced Vector eXtensions, 512 bits)
    • VNNI (Vector Neural Network Instructions)
    • TSX-NI (Transactional Synchronization eXtensions)
  • HPC interconnect: InfiniBand HDR100 (100 GBit/s)
  • Hostnames: logc0001 … logc0008
  • Accessible from outside as: lcluster13 … lcluster20.hrz.tu-darmstadt.de

MPI – Section (630 nodes)

  • 2x “Intel® Xeon® Platinum 9242 Processor”
  • 96 cores and 2x Intel® AVX-512 units per core
  • 384 GByte Main memory (DDR4-2933)
  • 48 cores per processor – Hyperthreading is off
    • CPU clock 2,3 GHz, Turbo 3,8 GHz
    • AVX-512 (Advanced Vector eXtensions, 512 bits)
    • VNNI (Vector Neural Network Instructions)
    • TSX-NI (Transactional Synchronization eXtensions)
  • HPC interconnect: InfiniBand HDR100 (100 GBit/s)
  • Hostnames: mpsc0001 … mpsc0630

MEM – Section (2 nodes)

  • 2x “Intel® Xeon® Platinum 9242 Processor”
  • 96 cores und 2x Intel® AVX-512 units per core
  • 1536 GByte Main memory (DDR4-2933)
  • 48 cores per processor – Hyperthreading is off
    • CPU clock 2,3 GHz, Turbo 3,8 GHz
    • AVX-512 (Advanced Vector eXtensions, 512 bits)
    • VNNI (Vector Neural Network Instructions)
    • TSX-NI (Transactional Synchronization eXtensions)
  • HPC interconnect: InfiniBand HDR100 (100 GBit/s)
  • Hostnames: mpqc0001 … mpqc0002

ACC – Section GPUs (8 nodes)

  • 4x “Intel® Xeon® Platinum 8260 Processor”
  • 96 cores and 2x Intel® AVX-512 units per core
  • 384 GByte Main memory (DDR4-2933)
  • 4x nodes with each 4x “NVIDIA® Tesla® V100” (Volta generation, GV100 chip)
  • 4x nodes with each 4x “NVIDIA® A100” (Ampere generation, GA100 chip)
  • 24 cores per processor – Hyperthreading is off
    • CPU clock 2,4 GHz, Turbo 3,9 GHz
    • AVX-512 (Advanced Vector eXtensions, 512 bits)
    • VNNI (Vector Neural Network Instructions)
    • TSX-NI (Transactional Synchronization eXtensions)
  • HPC interconnect: InfiniBand HDR100 (100 GBit/s)
  • Accelerator cards Nvidia Tesla V100:
    • 5120 CUDA cores
    • 640 Tensor cores
    • 32 GB CoWoS-HBM2 ECC RAM
    • 900 GByte/s Memory bandwidth
    • Double-Precision (64bit): 7 TFlop/s
    • Single-Precision (32bit): 14 TFlop/s
    • Tensor Performance: 112 TFlop/s
    • Hostnames: gvqc0001 … gvqc0004
  • Accelerator cards Nvidia A100:
    • 6912 CUDA cores
    • 432 Tensor cores
    • 40 GB CoWoS-HBM2 ECC RAM
    • 1600 GByte/s Memory bandwidth
    • Double-Precision (64bit): 9.7 TFlop/s (19.5 TFlop/s non Std.)
    • Single-Precision (32bit): 19.5 TFlop/s (156 TFlop/s non Std.)
    • Tensor Performance: 312 TFlop/s (624 TFlop/s with Sparsity)
    • Hostnames: gaqc0001 … gaqc0004

ACC – Section DGX A100 (3 nodes)

  • 2x “AMD EPYC™ 7742” Processor
  • 128 cores and 2x AVX-2 units per core
  • 1024 GByte Main memory (DDR4-3200)
  • 8x “NVIDIA® A100 Tensor Core GPUs” (Ampere generation)
  • 64 cores per processor
    • CPU clock 2,25 GHz, Boost 3,4 GHz
    • AVX-2 (Advanced Vector eXtensions, 256 bits)
    • PCIe® 4.0 x 128
  • Accelerator cards Nvidia A100:
    • 6912 CUDA cores
    • 432 Tensor cores
    • 40 GB CoWoS-HBM2 ECC RAM
    • 1600 GByte/s Memory bandwidth
    • Double-Precision (64bit): 9.7 TFlop/s (19.5 TFlop/s non Std.)
    • Single-Precision (32bit): 19.5 TFlop/s (156 TFlop/s non Std.)
    • Tensor Performance: 312 TFlop/s (624 TFlop/s with Sparsity)
  • HPC interconnect: 2x InfiniBand HDR200 (200 GBit/s)
  • Hostnames: gaoc0001 … gaoc0003

Totals (all compute and login nodes)

  • 62.592 cores
  • 257 TByte RAM
  • 16x Nvidia Tesla V100
  • 40x Nvidia A100