Tags:
create new tag
, view all tags

HPC Primer Hardware Configuration

As of Jan 1st, 2013

Compute Nodes Configuration

  • All the compute nodes are connected to the 1Gbps TCP/IP network
  • Some are connected to a 40Gbps InfiniBand fabric

Rack No. of No. of No. of Mem per Processor type No. of nodes No. of CPUs
No. Nodes CPUs/Node CPUs/Rack Node (AMD Opteron) on InfiniBand
0 25 4 100 8 GB 2x 2c 285 SE @ 2.8 GHz - -
  4 64 256 256 GB 2 x 32c 6274 @ 2.2 GHz - -
1 32 8 256 16 GB 2x 4c 2376 @ 2.3 GHz - -
2 34 8 272 16 GB 2x 4c 2376 @ 2.3 GHz - -
3 34 8 272 16 GB 2x 4c 2376 @ 2.3 GHz - -
4 33 8   16 GB 2x 4c 2376 @ 2.3 GHz - -
  1 4 268 8 GB 2x 2c 285 SE @ 2.8 GHz - -
5 34 8   16 GB 2x 4c 2376 @ 2.3 GHz - -
  1 4 276 8 GB 2x 2c 285 SE @ 2.8 GHz - -
6 32 8   16 GB 2x 4c 2376 @ 2.3 GHz - -
  1 4 268 8 GB 2x 2c 285 SE @ 2.8 GHz - -
7 25 16 400 32 GB 2x 8c 6128 @ 2.0 GHz 25 400
8 24 12 288 32 GB 2x 6c 4184 @ 2.8 GHz 24 288
9 11 24 264 64 GB 2x 12c 6180SE @ 2.5 GHz 11 264
10 6 64 384 256 GB 2 x 32c 6274 @ 2.2 GHz 6 384
Total: 296   3296 8.0 TB   66 1336

Disk Space Configuration

The disk use primer page gives detailed information on what disks to use and how to copy data from/to hydra.

  • The available disk space is served by two NetApp filers and two Dell servers
  • These are served to the compute nodes via NFS on the 1Gbps TCP/IP network
  • A PVFS2 filesystem is in the works to provide a parallel filessystem built using the IB fabric

IB Fabric Configuration

  • Current configuration: head node and 66 compute nodes for 1336 cores, via 3 switches.
sw # Port Hostname # sw # Port Hostname # sw # Port Hostname #
1 2 compute-10-2   2 1 compute-8-5   3 9 compute-9-10  
1 3 compute-8-22   2 2 hydra-2   3 10 compute-9-9  
1 4 compute-8-7   2 4 compute-8-3   3 11 compute-9-8  
1 5 compute-8-21   2 5 compute-8-4   3 12 compute-9-7  
1 6 compute-8-20   2 6 compute-8-1   3 13 compute-9-6  
1 7 compute-8-19   2 7 compute-8-2   3 14 compute-9-5  
1 8 compute-8-18   2 9 compute-8-0   3 15 compute-9-4  
1 9 compute-8-17   2 12 compute-7-0   3 16 compute-9-3  
1 10 compute-8-16   2 14 compute-7-4   3 17 compute-9-2  
1 11 compute-8-15   2 15 compute-7-7   3 18 compute-9-1  
1 12 compute-8-23   2 16 compute-7-10   3 19 compute-9-0  
1 13 compute-8-6   2 17 compute-7-11   3 20 compute-10-3  
1 14 compute-8-14   2 18 compute-7-9   3 21 compute-10-5  
1 15 compute-8-11   2 19 compute-7-19   3 22 compute-10-4  
1 16 compute-8-10   2 20 compute-7-18   3 23 compute-10-0  
1 17 compute-8-9   2 29 compute-7-23   3 24 compute-10-1 16
1 18 compute-8-8   2 30 compute-7-8  
1 19 compute-8-13   2 31 compute-7-22  
1 20 compute-8-12   2 32 compute-7-20  
1 21 compute-7-17   2 33 compute-7-21  
1 22 compute-7-16   2 34 compute-7-3  
1 23 compute-7-15   2 35 compute-7-2  
1 24 compute-7-12   2 36 compute-7-1 23  
1 25 compute-7-13  
1 26 compute-7-5  
1 27 compute-7-6  
1 28 compute-7-14  
1 30 compute-7-24 28  
switch 1 <==> 2: 7 ports   switch 2 <==> 3: 6 ports

-- SylvainKorzennikHPCAnalyst - 27 Jun 2012

Edit | Attach | Print version | History: r11 < r10 < r9 < r8 < r7 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r10 - 2013-07-11 - SylvainKorzennikHPCAnalyst
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2015 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback