HPC Interconnect
In all High Performance and Supercomputer systems, the HPC interconnect is of paramount importance. In contrast to server farms or cloud computing, the interconnect provides fast, low-latency and high throughput communication between all compute nodes.
MPI programs use the interconnect to utilize many compute nodes in parallel for large and complex calculations. By means of the interconnect, the MPI tasks convey data, states and intermediary results among each other.
Storage HDR
Infiniband HDR: The storage system (Home, Projects, Scratch) is the core of the interconnect fabric, and manages the whole system based on HDR: „High Data Rate“ 100 = 100 GBit/s (formerly EDR).
Lichtenberg II Cluster Stage 2
Infiniband HDR: the compute and login nodes of this cluster stage feature Mellanox ConnectX cards of the “HDR100” standard:
“High Data Rate 100” = 100 GBit/s.
While HDR in general offers up to 200 GBit/s, the nodes of this stage have cards with only 100 GBit/s – only the infiniband switches in between them communicate with HDR 200.
The internal IB topology of this cluster stage (“island”) is 1:1 non-blocking: any given (group of) compute nodes can communicate completely unharmed by any other (group of) compute node's network usage.
In other words, nowhere in the IB network of this stage is the number of interconnect links less than “one per compute node”.
Lichtenberg II Cluster Stage 1
Infiniband HDR: the compute and login nodes feature Mellanox ConnectX cards of the “HDR100” standard:
“High Data Rate” 100 = 100 GBit/s.
While HDR in general offers up to 200 GBit/s, the nodes of this stage have cards with only 100 GBit/s – only the infiniband switches in between them communicate with HDR 200.
The internal IB topology of this cluster stage (“island”) is 1:1 non-blocking: any given (group of) compute nodes can communicate completely unharmed by any other (group of) compute node's network usage.
In other words, nowhere in the IB network of this stage is the number of interconnect links less than “one per compute node”.
Further details on the Lichtenberg-HPC components can be found in our Hardware Overview .