- All the compute nodes are connected to the 1Gbps TCP/IP network
- Some are connected to a 40Gbps InfiniBand fabric

Rack | No. of | No. of | No. of | Mem per | Processor type | No. of nodes | No. of CPUs |
---|---|---|---|---|---|---|---|

No. | Nodes | CPUs/Node | CPUs/Rack | Node | (AMD Opteron) | on InfiniBand | |

0 | 25 | 4 | 100 | 8 GB | 2x 2c 285 SE @ 2.8 GHz | - | - |

4 | 64 | 256 | 256 GB | 2 x 32c 6274 @ 2.2 GHz | - | - | |

1 | 32 | 8 | 256 | 16 GB | 2x 4c 2376 @ 2.3 GHz | - | - |

2 | 34 | 8 | 272 | 16 GB | 2x 4c 2376 @ 2.3 GHz | - | - |

3 | 34 | 8 | 272 | 16 GB | 2x 4c 2376 @ 2.3 GHz | - | - |

4 | 33 | 8 | 16 GB | 2x 4c 2376 @ 2.3 GHz | - | - | |

1 | 4 | 268 | 8 GB | 2x 2c 285 SE @ 2.8 GHz | - | - | |

5 | 34 | 8 | 16 GB | 2x 4c 2376 @ 2.3 GHz | - | - | |

1 | 4 | 276 | 8 GB | 2x 2c 285 SE @ 2.8 GHz | - | - | |

6 | 32 | 8 | 16 GB | 2x 4c 2376 @ 2.3 GHz | - | - | |

1 | 4 | 268 | 8 GB | 2x 2c 285 SE @ 2.8 GHz | - | - | |

7 | 25 | 16 | 400 | 32 GB | 2x 8c 6128 @ 2.0 GHz | 25 | 400 |

8 | 24 | 12 | 288 | 32 GB | 2x 6c 4184 @ 2.8 GHz | 24 | 288 |

9 | 11 | 24 | 264 | 64 GB | 2x 12c 6180SE @ 2.5 GHz | 11 | 264 |

10 | 6 | 64 | 384 | 256 GB | 2 x 32c 6274 @ 2.2 GHz | 6 | 384 |

Total: | 296 | 3296 | 8.0 TB | 66 | 1336 |

The disk use primer page gives detailed information on what disks to use
and how to copy data from/to `hydra`

.

- The available disk space is served by two NetApp filers and two Dell servers
- These are served to the compute nodes via NFS on the 1Gbps TCP/IP network
- A
`PVFS2`

filesystem is*in the works*to provide a parallel filessystem built using the IB fabric

- Current configuration: head node and 66 compute nodes for 1336 cores, via 3 switches.

sw # | Port | Hostname | # | sw # | Port | Hostname | # | sw # | Port | Hostname | # |
---|---|---|---|---|---|---|---|---|---|---|---|

1 | 2 | compute-10-2 | 2 | 1 | compute-8-5 | 3 | 9 | compute-9-10 | |||

1 | 3 | compute-8-22 | 2 | 2 | hydra-2 | 3 | 10 | compute-9-9 | |||

1 | 4 | compute-8-7 | 2 | 4 | compute-8-3 | 3 | 11 | compute-9-8 | |||

1 | 5 | compute-8-21 | 2 | 5 | compute-8-4 | 3 | 12 | compute-9-7 | |||

1 | 6 | compute-8-20 | 2 | 6 | compute-8-1 | 3 | 13 | compute-9-6 | |||

1 | 7 | compute-8-19 | 2 | 7 | compute-8-2 | 3 | 14 | compute-9-5 | |||

1 | 8 | compute-8-18 | 2 | 9 | compute-8-0 | 3 | 15 | compute-9-4 | |||

1 | 9 | compute-8-17 | 2 | 12 | compute-7-0 | 3 | 16 | compute-9-3 | |||

1 | 10 | compute-8-16 | 2 | 14 | compute-7-4 | 3 | 17 | compute-9-2 | |||

1 | 11 | compute-8-15 | 2 | 15 | compute-7-7 | 3 | 18 | compute-9-1 | |||

1 | 12 | compute-8-23 | 2 | 16 | compute-7-10 | 3 | 19 | compute-9-0 | |||

1 | 13 | compute-8-6 | 2 | 17 | compute-7-11 | 3 | 20 | compute-10-3 | |||

1 | 14 | compute-8-14 | 2 | 18 | compute-7-9 | 3 | 21 | compute-10-5 | |||

1 | 15 | compute-8-11 | 2 | 19 | compute-7-19 | 3 | 22 | compute-10-4 | |||

1 | 16 | compute-8-10 | 2 | 20 | compute-7-18 | 3 | 23 | compute-10-0 | |||

1 | 17 | compute-8-9 | 2 | 29 | compute-7-23 | 3 | 24 | compute-10-1 | 16 | ||

1 | 18 | compute-8-8 | 2 | 30 | compute-7-8 | ||||||

1 | 19 | compute-8-13 | 2 | 31 | compute-7-22 | ||||||

1 | 20 | compute-8-12 | 2 | 32 | compute-7-20 | ||||||

1 | 21 | compute-7-17 | 2 | 33 | compute-7-21 | ||||||

1 | 22 | compute-7-16 | 2 | 34 | compute-7-3 | ||||||

1 | 23 | compute-7-15 | 2 | 35 | compute-7-2 | ||||||

1 | 24 | compute-7-12 | 2 | 36 | compute-7-1 | 23 | |||||

1 | 25 | compute-7-13 | |||||||||

1 | 26 | compute-7-5 | |||||||||

1 | 27 | compute-7-6 | |||||||||

1 | 28 | compute-7-14 | |||||||||

1 | 30 | compute-7-24 | 28 | ||||||||

switch 1 <==> 2: 7 ports | switch 2 <==> 3: 6 ports |

-- SylvainKorzennikHPCAnalyst - 27 Jun 2012

Topic revision: r10 - 2013-07-11 - SylvainKorzennikHPCAnalyst

Copyright © 2008-2015 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.

Ideas, requests, problems regarding TWiki? Send feedback

Ideas, requests, problems regarding TWiki? Send feedback