[ewg] Fwd: OpenSM for dual Rail

[Hal Rosenstock]

Hi Atul,

On 7/26/2014 5:45 AM, Atul Yadav wrote:
> Dear Admin,
> 
> In our exiting setup we are having 18 server with dual IB cards.
> And 2 IB switch.
> 
> We are planning to use MPI traffic on IB Port1 and Lustre Traffic on IB
> port 2.
> 
> IB Port 1 		Hostname 		IB Port 2
> IB Switch  1 	IP Address 	Hostname 	IP Address 	IB Switch  2
> Port 1 	192.168.2.1 	Compute-1 	192.168.3.1 	Port 1
> Port 2 	192.168.2.2 	Compute-2 	192.168.3.2 	Port 2
> Port 3 	192.168.2.3 	Compute-3 	192.168.3.3 	Port 3
> Port 4 	192.168.2.4 	Compute-4 	192.168.3.4 	Port 4
> Port 5 	192.168.2.5 	Compute-5 	192.168.3.5 	Port 5
> Port 6 	192.168.2.6 	Compute-6 	192.168.3.6 	Port 6
> Port 7 	192.168.2.7 	Compute-7 	192.168.3.7 	Port 7
> Port 8 	192.168.2.8 	Compute-8 	192.168.3.8 	Port 8
> Port 9 	192.168.2.9 	Compute-9 	192.168.3.9 	Port 9
> Port 10 	192.168.2.10 	Compute-10 	192.168.3.10 	Port 10
> Port 11 	192.168.2.11 	Compute-11 	192.168.3.11 	Port 11
> Port 12 	192.168.2.12 	Compute-12 	192.168.3.12 	Port 12
> Port 13 	192.168.2.13 	Compute-13 	192.168.3.13 	Port 13
> Port 14 	192.168.2.14 	Compute-14 	192.168.3.14 	Port 14
> Port 15 	192.168.2.15 	Compute-15 	192.168.3.15 	Port 15
> Port 16 	192.168.2.16 	Compute-16 	192.168.3.16 	Port 16
> Port 17 	192.168.2.17 	Compute-17 	192.168.3.17 	Port 17
> Port 18 	192.168.2.18 	Compute-18 	192.168.3.18 	Port 18
> 
> 
> 
> need your help in opensm configuration and setup ....

You need an SM per subnet. They should be setup to have different subnet
prefixes.

Do you plan on running OpenSM for each subnet on the same or different
machines ?

It's a little simpler if it's different machines.

If it's on same machine, basically you need 2 options files (one for
each subnet/OpenSM) and make the output (/var/log and /var/cache/opensm
folders) go to separate files/directories. You can do that with the
various config options.

Also, see:
https://www.mail-archive.com/linux-rdma@vger.kernel.org/msg03559.html

I don't see OSM_LOG_DIR in recent OpenSMs but there is log_prefix option.

-- Hal

> Thank You
> Atul Yadav
> 
> 
> 
> _______________________________________________
> ewg mailing list
> ewg at lists.openfabrics.org
> http://lists.openfabrics.org/mailman/listinfo/ewg