Hardware
Lichtenberg I Phase II

Hardware of Phase II of the Lichtenberg I

(in operation from 2015 to May 2021)

Login (8 nodes)

  • 2x Processors „Intel® Xeon® Processor E5-2680 v3“
    • 12 cores per processor
    • 1 AVX2 unit (2.1 GHz plus Turbo-Mode)
  • 24 cores in total per node – Hyperthreading is off
  • Processor clock 2,5 GHz – with Turbo of up to 3,3 GHz (when using fewer cores)
  • 128 GByte main memory per node
  • Network: 1x FDR-14 InfiniBand and 2x 10GBit-Ethernet
  • Hostnames: hlb0001 … hlb0008
  • Accessible from outside as:
    lcluster5 … lcluster12 (append .hrz.tu-darmstadt.de if necessary)

MPI2 – section (596 nodes)

1x island (i19): 84 nodes (2016 cores, 5376 GByte main memory, 64 GByte per node)

16x islands (i20-i35): each with 32 nodes (768 cores, 2048 GByte main memory, 64 GByte per node)

1x island (nvd2): 23 nodes (552 cores, 1472 GByte main memory, 64 GByte per node)

  • 2x Processors „Intel® Xeon® Processor E5-2680 v3“
    • 12 cores per processor
    • Hyperthreading is off
    • one AVX2 unit per core (2.1 GHz plus Turbo)
  • 24 cores in total per node
  • Processor clock 2,5 GHz – with Turbo of up to 3,3 GHz (when using fewer cores)
  • 64 GByte main memory per node
  • Network: 1x FDR-14 InfiniBand and 1x 1GBit-Ethernet
    • All nodes inside an island: 1:1 blocking (MPI)
    • Across islands: ca. 1:8 blocking (MPI)
  • Hostnames: hpb0001 … hpb0596

MEM2 – section (3 nodes)

4x nodes: each 60 cores (with AVX) and 1024 GByte main memory per node

  • 4 x Processors „Intel® Xeon® Processor E7-4890 v2“
    • 15 cores per processor
    • Hyperthreading is off
    • one AVX unit per core
  • 60 cores in total per node
  • Processor clock 2,8 GHz – with Tubo of up to 3,4 GHz
  • 1024 GByte main memory per node
  • Network: 2 x FDR-14 InfiniBand and 1 x 10GBit-Ethernet
  • Hostnames: heb0001 … heb0004

ACC2 (GPU) – Section (32 nodes)

2x (nvd4) – nodes: each 24 cores, 64 GByte main memory and each 2x “NVIDIA® Tesla™ K40m”

1x (nvd8) – node: also 24 cores, 64 GByte main memory and 2x “NVIDIA® Tesla™ K80 – Dual-GPU”

(Remaining nodes see details)

  • 2 x Processors „Intel® Xeon® Processor E5-2680 v3“
  • Processors, main memory and network similar to MPI2 – Section
  • 2 x accelerator cards per node
  • 2x nodes with two “NVIDIA® Tesla™ K40m” each
    • ca. 1,43 TFlop/s performance (double precision, peak – theoretical)
    • 2880 stream cores
    • 12 GByte memory
    • 288 GByte/s memory bandwidth (peak – theoretical)
    • Hostnames: hab0001 … hab0002
  • 1x node with two “NVIDIA® Tesla™ K80 – Dual-GPU”
    • ca. 2,91 TFlop/s performance each (double precision, peak – theoretiical)
    • 4992 stream cores
    • 24 GByte memory (2x 12 GByte)
    • 480 GByte/s memory bandwidth (peak – theoretical)
    • Hostnames: hab0003
  • The remaining 29x nodes without accelerator cards, thus usable similar to MPI
    • Hostnames: hab0004 … hab0032

Further reading:

Explanation of the "island topology " of the Lichtenberg I