[ewg] physically separate subnets (was: OFED March 24 meeting summary on OFED 1.4 plans)

[Hal Rosenstock]

On Thu, 2008-04-03 at 10:47 -0400, Jeff Squyres wrote:
> In Open MPI, we require physically different ("air gapped") subnets to  
> have different subnet ID's so that we can compute reachability  
> correctly.

Don't understand what the "air gapped" reference means.

> I don't know how to do it otherwise.

Me neither.

-- Hal

> 
> 
> On Apr 3, 2008, at 10:40 AM, Tang, Changqing wrote:
> >
> > You have a system, all HCAs have two ports, all port 1 are connected  
> > to the first switch,
> > all port 2 are connected to the second switch, there is NO link  
> > between the two switches.
> > We call this system has two physically separated fabrics. If you  
> > have a bridge link
> > between the two switches, then it becomes a single fabric.
> >
> > The same thing for multiple HCAs on nodes.
> >
> > The problem is, from MPI side, (and by default), we don't know which  
> > port is on which
> > fabric, since the subnet prefix is the same. We rely on system admin  
> > to config two
> > different subnet prefixes for HP-MPI to work.
> >
> > No vendor has claimed to support this.
> >
> > --CQ
> >
> >> -----Original Message-----
> >> From: Hal Rosenstock [mailto:hrosenstock at xsigo.com]
> >> Sent: Thursday, April 03, 2008 9:32 AM
> >> To: Tang, Changqing
> >> Cc: Erez Zilber; Tziporet Koren; ewg at lists.openfabrics.org;
> >> general at lists.openfabrics.org
> >> Subject: RE: [ofa-general] Re: [ewg] OFED March 24 meeting
> >> summary on OFED 1.4 plans
> >>
> >> CQ,
> >>
> >> On Thu, 2008-04-03 at 14:27 +0000, Tang, Changqing wrote:
> >>> Can we address multiple-fabrics (physically separated) support ?
> >>
> >> Can you elaborate on what you mean by "physically separated" ?
> >>
> >> -- Hal
> >>
> >>>
> >>>
> >>> --CQ Tang
> >>>
> >>>> -----Original Message-----
> >>>> From: general-bounces at lists.openfabrics.org
> >>>> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Erez
> >>>> Zilber
> >>>> Sent: Thursday, April 03, 2008 8:51 AM
> >>>> To: Tziporet Koren
> >>>> Cc: ewg at lists.openfabrics.org; general at lists.openfabrics.org
> >>>> Subject: [ofa-general] Re: [ewg] OFED March 24 meeting summary on
> >>>> OFED 1.4 plans
> >>>>
> >>>>>
> >>>>> *OFED 1.4:*
> >>>>> 1. Kernel base: since we target 1.4 release to Sep we
> >> target the
> >>>>> kernel base to be 2.6.27
> >>>>>    This is a good target, but we may need to stay with
> >>>> 2.6.26 if the
> >>>>> kernel progress will not be aligned.
> >>>>>
> >>>>> 2. Suggestions for new features:
> >>>>>
> >>>>>    * NFS-RDMA
> >>>>>    * Verbs: Reliable Multicast (to be presented at Sonoma)
> >>>>>    * SDP - Zero copy (There was a question on IPv6 support
> >>>> - seems no
> >>>>>      one interested for now)
> >>>>>    * IPoIB - continue with performance enhancements
> >>>>>    * Xsigo new virtual NIC
> >>>>>    * New vendor HW support - non was reported so far (IBM
> >>>> and Chelsio
> >>>>>      - do you have something?)
> >>>>>    * OpenSM:
> >>>>>          o Incremental routing
> >>>>>          o Temporary SA DB - to answer queries and a heavy
> >>>> sweep is done
> >>>>>          o APM - disjoint paths (?)
> >>>>>          o MKey manager (?)
> >>>>>          o Sasha to send more management features
> >>>>>    * MPI:
> >>>>>          o Open MPI 1.3
> >>>>>          o APM support in MPI
> >>>>>          o mvapich ???
> >>>>>    * uDAPl
> >>>>>          o Extensions for new APIs (like XRC) - ?
> >>>>>          o uDAPL provider for interop between Windows & Linux
> >>>>>          o 1.2 and 2.0 will stay
> >>>>>
> >>>>
> >>>> As I wrote in an earlier discussion (~2 months ago), we
> >> plan to add
> >>>> tgt (SCSI target) with iSCSI over iSER (and TCP of
> >>>> course) support. The git tree for tgt already exists on the ofa
> >>>> server.
> >>>>
> >>>> Erez
> >>>>
> >>>> _______________________________________________
> >>>> general mailing list
> >>>> general at lists.openfabrics.org
> >>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >>>>
> >>>> To unsubscribe, please visit
> >>>> http://openib.org/mailman/listinfo/openib-general
> >>>>
> >>> _______________________________________________
> >>> general mailing list
> >>> general at lists.openfabrics.org
> >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >>>
> >>> To unsubscribe, please visit
> >>> http://openib.org/mailman/listinfo/openib-general
> >>
> >>
> > _______________________________________________
> > ewg mailing list
> > ewg at lists.openfabrics.org
> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
> 
>