[ewg] Re: [ofa-general] RE: Agenda for the OFED meeting today (Jan 5, 09)

[Jeff Squyres]

Hmm.  Perhaps I'm not grokking your answer -- did you answer my  
question?  I'm indirectly asking about scalability of the SM to have  
hundreds/thousands of MPI processes simultaneously querying the SM.


On Jan 5, 2009, at 11:47 AM, John Russo wrote:

> When using routing algorithms such as lash, the SL used per end to  
> end connection will vary based on the route.  lash uses multiple VLs  
> to avoid credit loops.  As such the SL reported will vary based on  
> fabric topology and which pair of end nodes the path is being  
> requested on behalf of.
>
> -----Original Message-----
> From: Jeff Squyres [mailto:jsquyres at cisco.com]
> Sent: Monday, January 05, 2009 11:39 AM
> To: John Russo
> Cc: Tziporet Koren; ewg at lists.openfabrics.org; general at lists.openfabrics.org
> Subject: Re: [ofa-general] RE: Agenda for the OFED meeting today  
> (Jan 5, 09)
>
> Would all MPI processes need to query the SM for each path that they
> want to use in a QP?
>
>
> On Jan 5, 2009, at 11:31 AM, John Russo wrote:
>
>> Another suggestion for 1.5
>>
>> Implementation of SA queries for Path Records (using IBTA 1.2.1
>> ServiceId field) in all OFED ULPs, especially for MPI
>>            The IBTA standard defines that the proper way to
>> establish a connection is to get a PathRecord from the SM/SA and use
>> it to define all the attributes of the communication path.
>> Ideally the IBTA CM should then be used to establish the connection
>> and QPs as well.
>>
>> At present, openmpi, mvapich1 and mvapich2 do not use PathRecords,
>> but instead hard code attributes like the PKey, SL, etc.
>> In some cases these hardcoded values can be overridden by
>> configurable values such as PKey and SL, but such values must be
>> uniform across all connections and must be provided per job (which
>> can be error prone/tedious).
>>
>>            At present opensm supports PKeys and SLs, however MPI
>> cannot easily use these features.
>> Other features, such as lash routing, in opensm do not work properly
>> with MPI because the SL must be uniform across all connections, but
>> for lash it will vary per route.
>>
>> Additionally, applications which do not use PathRecords will have
>> difficulties with advanced features like IB routing, partitioning,
>> etc.  All of which are available or being worked on in opensm.
>>
>> From: ewg-bounces at lists.openfabrics.org [mailto:ewg-bounces at lists.openfabrics.org
>> ] On Behalf Of Tziporet Koren
>> Sent: Monday, January 05, 2009 1:00 AM
>> To: ewg at lists.openfabrics.org
>> Cc: general at lists.openfabrics.org
>> Subject: [ewg] Agenda for the OFED meeting today (Jan 5, 09)
>>
>>
>> Hello all,
>>
>> I hope we all had nice holidays and vacations, and now it's the time
>> to get back to business.
>>
>> Agenda for OFED meeting today:
>>
>> 1. Conclusions from OFED 1.4 release: Open discussion
>>
>> 2. Do we wish to have OFED 1.4.: Please send pros & cons before the
>> meeting
>>
>> 3. OFED 1.5: Schedule and features.
>>
>> This is what we presented in SC08 about 1.5:
>>
>> Preliminary Schedule:
>>
>> 	* Feature Freeze: 3/20/09
>> 	* Alpha Release: 3/20/09
>> 	* Beta Release: 4/20/09
>> 	* RC1: 5/5/09
>> 	* RC2-RCx: About every 2 weeks as needed
>> 	* Release: June 2009
>> Features:
>>
>> 	* Kernel.org: 2.6.28 and 2.6.29
>> 	* Multiple Event Queues to support Multi-core CPUs
>> 	* NFS/RDMA - GA
>> 	* RDS support for iWARP
>> 	* OpenMPI 1.3
>> 	* Add support/backports for RedHat EL 5.3 and EL 4.8, SLES 11
>> 	* Support for Mellanox vNIC (EoIB) and FCoIB with BridgeX device
>> 	* more TBD...
>>
>> We also presented the OS matrix but I suggest we will close this in
>> the next meeting.
>>
>> My proposal:
>>
>> 	* Have the release in July and not June - so we will have more time
>> for development
>> 	* Stick to one kernel version base and not change in the middle
>> since we saw that changing the kernel base caused a delay.
>> We need to decide in the meeting if it is 2.6.29 or we should wait
>> for 2.6.30.
>> 	* Add IB over Eth - this is similar to iWARP but more like IB (e.g.
>> including UD), and can work over ConnectX.
>>
>> Please send your suggestions to the list before the meeting if
>> possible
>>
>> Tziporet
>>
>>
>> _______________________________________________
>> general mailing list
>> general at lists.openfabrics.org
>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>
>> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>
>
> -- 
> Jeff Squyres
> Cisco Systems
>


-- 
Jeff Squyres
Cisco Systems