[ofa-general] physically separate subnets (was: OFED March 24 meeting summary on OFED 1.4 plans)

[Jeff Squyres]

In Open MPI, we require physically different ("air gapped") subnets to  
have different subnet ID's so that we can compute reachability  
correctly.  I don't know how to do it otherwise.


On Apr 3, 2008, at 10:40 AM, Tang, Changqing wrote:
>
> You have a system, all HCAs have two ports, all port 1 are connected  
> to the first switch,
> all port 2 are connected to the second switch, there is NO link  
> between the two switches.
> We call this system has two physically separated fabrics. If you  
> have a bridge link
> between the two switches, then it becomes a single fabric.
>
> The same thing for multiple HCAs on nodes.
>
> The problem is, from MPI side, (and by default), we don't know which  
> port is on which
> fabric, since the subnet prefix is the same. We rely on system admin  
> to config two
> different subnet prefixes for HP-MPI to work.
>
> No vendor has claimed to support this.
>
> --CQ
>
>> -----Original Message-----
>> From: Hal Rosenstock [mailto:hrosenstock at xsigo.com]
>> Sent: Thursday, April 03, 2008 9:32 AM
>> To: Tang, Changqing
>> Cc: Erez Zilber; Tziporet Koren; ewg at lists.openfabrics.org;
>> general at lists.openfabrics.org
>> Subject: RE: [ofa-general] Re: [ewg] OFED March 24 meeting
>> summary on OFED 1.4 plans
>>
>> CQ,
>>
>> On Thu, 2008-04-03 at 14:27 +0000, Tang, Changqing wrote:
>>> Can we address multiple-fabrics (physically separated) support ?
>>
>> Can you elaborate on what you mean by "physically separated" ?
>>
>> -- Hal
>>
>>>
>>>
>>> --CQ Tang
>>>
>>>> -----Original Message-----
>>>> From: general-bounces at lists.openfabrics.org
>>>> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Erez
>>>> Zilber
>>>> Sent: Thursday, April 03, 2008 8:51 AM
>>>> To: Tziporet Koren
>>>> Cc: ewg at lists.openfabrics.org; general at lists.openfabrics.org
>>>> Subject: [ofa-general] Re: [ewg] OFED March 24 meeting summary on
>>>> OFED 1.4 plans
>>>>
>>>>>
>>>>> *OFED 1.4:*
>>>>> 1. Kernel base: since we target 1.4 release to Sep we
>> target the
>>>>> kernel base to be 2.6.27
>>>>>    This is a good target, but we may need to stay with
>>>> 2.6.26 if the
>>>>> kernel progress will not be aligned.
>>>>>
>>>>> 2. Suggestions for new features:
>>>>>
>>>>>    * NFS-RDMA
>>>>>    * Verbs: Reliable Multicast (to be presented at Sonoma)
>>>>>    * SDP - Zero copy (There was a question on IPv6 support
>>>> - seems no
>>>>>      one interested for now)
>>>>>    * IPoIB - continue with performance enhancements
>>>>>    * Xsigo new virtual NIC
>>>>>    * New vendor HW support - non was reported so far (IBM
>>>> and Chelsio
>>>>>      - do you have something?)
>>>>>    * OpenSM:
>>>>>          o Incremental routing
>>>>>          o Temporary SA DB - to answer queries and a heavy
>>>> sweep is done
>>>>>          o APM - disjoint paths (?)
>>>>>          o MKey manager (?)
>>>>>          o Sasha to send more management features
>>>>>    * MPI:
>>>>>          o Open MPI 1.3
>>>>>          o APM support in MPI
>>>>>          o mvapich ???
>>>>>    * uDAPl
>>>>>          o Extensions for new APIs (like XRC) - ?
>>>>>          o uDAPL provider for interop between Windows & Linux
>>>>>          o 1.2 and 2.0 will stay
>>>>>
>>>>
>>>> As I wrote in an earlier discussion (~2 months ago), we
>> plan to add
>>>> tgt (SCSI target) with iSCSI over iSER (and TCP of
>>>> course) support. The git tree for tgt already exists on the ofa
>>>> server.
>>>>
>>>> Erez
>>>>
>>>> _______________________________________________
>>>> general mailing list
>>>> general at lists.openfabrics.org
>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>>>
>>>> To unsubscribe, please visit
>>>> http://openib.org/mailman/listinfo/openib-general
>>>>
>>> _______________________________________________
>>> general mailing list
>>> general at lists.openfabrics.org
>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>>
>>> To unsubscribe, please visit
>>> http://openib.org/mailman/listinfo/openib-general
>>
>>
> _______________________________________________
> ewg mailing list
> ewg at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg


-- 
Jeff Squyres
Cisco Systems