Tags:
create new tag
, view all tags

HPC Primer Hardware Configuration

As of Jan 1st, 2013

Compute Nodes Configuration

  • All the compute nodes are connected to the 1Gbps TCP/IP network
  • Some are connected to a 40Gbps InfiniBand fabric

Rack No. of No. of No. of Mem per Processor type No. of nodes No. of CPUs
No. Nodes CPUs/Node CPUs/Rack Node (AMD Opteron) on InfiniBand
0 25* 4 100 8 GB 2x 2c 285 SE @ 2.8 GHz - -
  4 64 256 256 GB 2 x 32c 6274 @ 2.2 GHz - -
1 32 8 256 16 GB 2x 4c 2376 @ 2.3 GHz - -
2 34 8 272 16 GB 2x 4c 2376 @ 2.3 GHz - -
3 34 8 272 16 GB 2x 4c 2376 @ 2.3 GHz - -
4 33 8   16 GB 2x 4c 2376 @ 2.3 GHz - -
  1* 4 268 8 GB 2x 2c 285 SE @ 2.8 GHz - -
5 34 8   16 GB 2x 4c 2376 @ 2.3 GHz - -
  1* 4 276 8 GB 2x 2c 285 SE @ 2.8 GHz - -
6 32 8   16 GB 2x 4c 2376 @ 2.3 GHz - -
  1* 4 268 8 GB 2x 2c 285 SE @ 2.8 GHz - -
7 25 16 400 32 GB 2x 8c 6128 @ 2.0 GHz 25 400
8 24 12 288 32 GB 2x 6c 4184 @ 2.8 GHz 24 288
9 11 24 264 64 GB 2x 12c 6180SE @ 2.5 GHz 11 264
10 6 64 384 256 GB 2 x 32c 6274 @ 2.2 GHz 6 384
Total: 296   3296 8.0 TB   66 1336
  * these 28 nodes (oldest) are in the process of being decommissioned

Disk Space Configuration

The disk use primer page gives detailed information on what disks to use and how to copy data from/to hydra.

  • The available disk space is served by two NetApp filers and two Dell servers
  • These are served to the compute nodes via NFS on the 1Gbps TCP/IP network
  • A PVFS2 filesystem is in the works to provide a parallel filessystem built using the IB fabric

IB Fabric Configuration

  • Current configuration: head node and 66 compute nodes for 1336 cores, via 3 switches.
sw # Port Hostname # sw # Port Hostname # sw # Port Hostname #
1 2 compute-10-2   2 1 compute-8-5   3 9 compute-9-10  
1 3 compute-8-22   2 2 hydra-2   3 10 compute-9-9  
1 4 compute-8-7   2 4 compute-8-3   3 11 compute-9-8  
1 5 compute-8-21   2 5 compute-8-4   3 12 compute-9-7  
1 6 compute-8-20   2 6 compute-8-1   3 13 compute-9-6  
1 7 compute-8-19   2 7 compute-8-2   3 14 compute-9-5  
1 8 compute-8-18   2 9 compute-8-0   3 15 compute-9-4  
1 9 compute-8-17   2 12 compute-7-0   3 16 compute-9-3  
1 10 compute-8-16   2 14 compute-7-4   3 17 compute-9-2  
1 11 compute-8-15   2 15 compute-7-7   3 18 compute-9-1  
1 12 compute-8-23   2 16 compute-7-10   3 19 compute-9-0  
1 13 compute-8-6   2 17 compute-7-11   3 20 compute-10-3  
1 14 compute-8-14   2 18 compute-7-9   3 21 compute-10-5  
1 15 compute-8-11   2 19 compute-7-19   3 22 compute-10-4  
1 16 compute-8-10   2 20 compute-7-18   3 23 compute-10-0  
1 17 compute-8-9   2 29 compute-7-23   3 24 compute-10-1 16
1 18 compute-8-8   2 30 compute-7-8  
1 19 compute-8-13   2 31 compute-7-22  
1 20 compute-8-12   2 32 compute-7-20  
1 21 compute-7-17   2 33 compute-7-21  
1 22 compute-7-16   2 34 compute-7-3  
1 23 compute-7-15   2 35 compute-7-2  
1 24 compute-7-12   2 36 compute-7-1 23  
1 25 compute-7-13  
1 26 compute-7-5  
1 27 compute-7-6  
1 28 compute-7-14  
1 30 compute-7-24 28  
switch 1 <==> 2: 7 ports   switch 2 <==> 3: 6 ports

-- SylvainKorzennikHPCAnalyst - 27 Jun 2012

Topic revision: r11 - 2013-11-01 - SylvainKorzennikHPCAnalyst
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2015 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback