Child pages
  • What is the storage and network topology of the RI Clusters?
Skip to end of metadata
Go to start of metadata

Background

This page describes that network topology of the RI Cluster Systems (including odin and tyr) and the Standalone RI systems (jord and idun). See the clusters KB label for a complete list of cluster-related documentation.

Ethernet

All cluster systems have standard 1Gb/s Ethernet connections with a 1Gb/s uplink to the rest of the SoIC and IU networks. These ethernet connections are to Nortel Baystack 5510 switches using a FAST (Flexible Advanced Stacking Technology) stack architecture that provides full wire speeds across the entire switch stack.

A limited number of the tyr cluster nodes also have additional 1Gb/s links to the IU Research network and there is a 10Gb/s uplink to this network. This provides access to the Data Capacitor from a limited number of tyr nodes.

The physical topology of these networks is shown below.

Storage Network

There are 2 file servers for use with the odin cluster:

Server

Capacity

Mount Point

Access Paths

rinfs1

3.2TB

rinfs1:/home (home directories)

/u/username (default home directory)

/l/san (software)

rinfs2

8TB

rinfs2:/home

/nfs1-data/username

/nfs/odinnfs1/san36-1-0/

InfiniBand Network

All 128 nodes of the Odin cluster have 8Gb/s connections to a single 144-port Mellanox Infiniband Switch. There are 12 12-port line cards in the switch the the ports are connected as follows:

Line Card

Port 1

Port 2

Port 3

Port 4

Port 5

Port 6

Port 7

Port 8

Port 9

Port 10

Port 11

Port 12

1:

odin036

odin037

odin038

odin039

odin040

odin041

odin001

odin002

odin003

odin004

odin005

odin006

2:

odin042

odin043

odin044

odin045

odin046

odin047

odin007

odin008

odin009

odin010

odin011

odin012

3:

odin048

odin049

odin050

odin051

odin052

odin053

odin013

odin014

odin015

odin016

odin017

odin018

4:

odin054

odin055

odin056

odin057

odin058

odin059

odin019

odin020

odin021

odin022

odin023

odin024

5:

odin060

odin061

odin062

odin063

odin064

odin065

odin025

odin026

odin027

odin028

odin029

odin030

6:

odin066

odin067

odin068

odin069

odin070

odin071

odin031

odin032

odin033

odin034

odin035

odin094

7:

odin072

odin073

odin074

odin075

odin076

odin077

odin095

odin096

odin097

odin098

odin099

odin100

8:

odin078

odin079

odin080

odin081

odin082

odin083

odin101

odin102

odin103

odin104

odin105

odin106

9:

odin084

odin085

odin086

odin087

odin088

odin089

odin107

odin108

odin109

odin110

odin111

odin112

10:

odin090

odin091

odin092

odin093

unused

unused

odin113

odin114

odin115

odin116

odin117

odin118

11:

unused

unused

unused

unused

unused

unused

odin119

odin120

odin121

odin122

odin123

odin124

12:

unused

unused

unused

unused

unused

unused

odin125

odin126

odin127

odin128

unused

odin (head)