Introduction of ITO
Update: 24 Dec. 2020Supercomputer system ITO started its service on January 2018. Based on Intel Xeon Gold (Skylake architecture) CPUs and NVIDIA Tesla P100 (Pascal architecture) GPUs, its theoretical peak performance reaches around 10PFLOPS.
This system is designed as a research infrastructure for both traditional computational science field and new data-science field. With a facility for detailed analysis on electric power consumption, and interfaces for interacting with public clouds, it aims to provide computational resources for more advanced supercomputing and broader research fields.
ITO is available not only for Japanese residents but also foreign researchers via international programs such as JHPCN and HPCI.
Hardware
Total System
| Theoretical Peak Performance | 10.43 PFLOPS (CPU：7.72 PFLOPS, GPU：2.71 PFLOPS) |
|---|---|
| Amount of Memory | 433 TB (DDR4 memory only) |
| Interconnect | Infiniband EDR 4x (100Gbps), Full Bisection Bandwidth Fat Tree |
| Amount of Storage | 24.6 PB |
Subsystem A
| Machine | Fujitsu PRIMERGY CX2550/CX2560 M4 | |
|---|---|---|
| Computing Node | CPU | Intel Xeon Gold 6154 (Skylake-SP) (3.0 GHz (Turbo 3.7 GHz), 18 core）x 2 / node |
| Theoretical Peak Performance | 3,456 GFLOPS / node (Double Precision) | |
| Amount of Memory | DDR4 192 GB / node | |
| Memory Bandwidth | 255.9 GB/sec / node | |
| Number of Nodes | 2,000 | |
| Total Number of Cores | 72,000 | |
| Total Theoretical Peak Performance | 6.91 PFLOPS (Double Precision) | |
| Total Amount of Memory | 384 TB | |
| Interconnect | InfiniBand EDR 4x (100Gbps) | |
| Local Storage of Node |
1TB HDD x 2 (SATA) Some nodes (CX2560) also consists of 800GB HDD x 1 (SATA) |
|
Subsystem B
| Machine | Fujitsu PRIMERGY CX2570 M4 | |
|---|---|---|
| Computing Node | CPU | Intel Xeon Gold 6140 (Skylake-SP) (2.3 GHz (Turbo 3.7 GHz), 18 core）x 2 / node |
| GPU | NVIDIA Tesla P100 (Pascal) (1,328 - 1,480 MHz, 56 SM (3584 CUDA core)) x 4 / node | |
| Theoretical Peak Performance | CPU : 2,649.6 GFLOPS / node (Double Precision) GPU : 5.3 TFLOPS (with Boost Clock) / 1GPU (Double Precision) |
|
| Amount of Memory | DDR4 : 384 GB / node HBM2 : 16 GB / 1GPU |
|
| Memory Bandwidth | DDR4 : 255.9 GB/sec / node HBM2 : 732 GB/sec / 1GPU |
|
| CPU-GPU Connection | PCI-Express Gen.3 x16 (16GB/sec) | |
| GPU-GPU Connection | NVLink (20GB/sec x 1or2) | |
| Number of Nodes | 128 | |
| Total Number of Cores | CPU : 4,608 GPU : 1,835,008 |
|
| Total Theoretical Peak Performance | CPU : 0.34 PFLOPS (Double Precision) GPU : 2.71 PFLOPS (Double Precision) |
|
| Total Amount of Memory | DDR4 : 49 TB HBM2 : 8.19 TB |
|
| Interconnect | InfiniBand EDR 4x (100Gbps) | |
| Local Storage of Node |
1TB HDD x 2 (SATA) 800GB HDD x 1 (SATA) |
|
Normal Nodes of Frontend
| Machine | HPE DL380 Gen10 | |
|---|---|---|
| Computing Node | CPU | Intel Xeon Gold 6140 (Skylake-SP) (2.3 GHz (Turbo 3.7 GHz), 18 core）x 2 / node |
| GPU | NVIDIA Quadro P4000 (Pascal) x 1 / node | |
| Theoretical Peak Performance | CPU : 2,649.6 GFLOPS / node | |
| Amount of Memory | DDR4 : 384 GB / node GDDR5 : 8GB |
|
| Memory Bandwidth | DDR4 : 255.9 GB/s / node GDDR5 : 243 GB/s |
|
| CPU-GPU Connection | PCI-Express Gen.3 x16 (16GB/sec) | |
| Number of Nodes | 160 | |
| Total Theoretical Peak Performance | CPU : 0.42 PFLOPS (Double Precision) | |
| Total Amount of Memory | DDR4 : 61 TB GDDR5 : 1.28 TB |
|
| Interconnect | 10Gb Ethernet x 1 | |
| Local Storage of Node | 2TB HDD x 2 (SAS) | |
| Other | Virtual hosts are available. | |
Large Nodes of Frontend
| Machine | SGI UV 300 | |
|---|---|---|
| Computing Node | CPU | Intel Xeon E7-8880 v4 (Broadwell-EX) (2.2 GHz, 22 core) x 16 / node |
| GPU | NVIDIA Quadro M4000 (Maxwell) x 1 / node | |
| Theoretical Peak Performance | CPU : 12.39 TFLOPS (Double Precision) / node | |
| Amount of Memory | DDR4 : 12 TB / node GDDR5 : 8 GB |
|
| Memory Bandwidth | DDR4 : 1,360 GB/s / node GDDR5 : 192 GB/s |
|
| CPU-GPU Connection | PCI-Express Gen.3 x16 (16GB/sec) | |
| Number of Nodes | 4 | |
| Total Theoretical Peak Performance | CPU : 49.6 TFLOPS (Double Precision) | |
| Total Amount of Memory | DDR4 : 48 TB GDDR5 : 32 GB |
|
| Interconnect | 10Gb Ethernet x 1 | |
| Local Storage of Node | 2TB HDD x 2 (SAS) | |
| Other | Virtual hosts are available. | |
Login Nodes
| Machine | Fujitsu PRIMERGY RX2540 M4 | |
|---|---|---|
| Computing Nodes | CPU | Intel Xeon Gold 6140 (Skylake-SP) (2.3 GHz (Turbo 3.7 GHz), 18 core）x 2 / node |
| Theoretical Peak Performance | 2,649.6 GFLOPS / node (Double Precision) | |
| Amount of Memory | DDR4 384 GB / node | |
| Memory Bandwidth | 255.9 GB/sec / node | |
| Number of Nodes | 2 | |
| Total Number of Cores | 72 | |
| Interconnect (with nodes in ITO) |
InfiniBand EDR 4x (100Gbps) x 2 10Gb Ethernet x 1 |
|
| Local Storage of Node | 1TB HDD x 2 (SATA) | |