[ewg] Re: [OMPI devel] [ofa-general] Re: OMPI over ofed udapl - bugs opened

[Jeff Squyres]

On May 10, 2007, at 9:02 AM, Or Gerlitz wrote:

>>> A different approach which you might want to consider is to have  
>>> at the btl level --two-- connections per <src,dst> ranks. so if A  
>>> wants to send B it does so through the A --> B connection and if  
>>> B wants to send A it does so through the B --> A connection. To  
>>> some extent, this is the approach taken by IPoIB-CM (I am not  
>>> enough into the RFC to understand the reasoning but i am quite  
>>> sure this was the approach in the initial implementation). At  
>>> first thought it mights seems not very elegant, but taking it  
>>> into the details (projected on the ompi env) you might find it   
>>> even nice.
>> What is the advantage of this approach?
>
> To start with, my hope here is at least to be able play defensive  
> here, that is convince you that the disadvantages are minor, where  
> only if this fails, would schedule myself some reading into the  
> ipoib-cm rfc to dig the advantages.

I ask about the advantages because OMPI currently treats QP's as bi- 
directional.  Having OMPI treat them at unidirectional would be a  
change.  I'm not against such a change, but I think we'd need to be  
convinced that there are good reasons to do so.  For example, on the  
surface, it seems like this scheme would simply consume more QPs and  
potentially more registered memory (and is therefore unattractive).

-- 
Jeff Squyres
Cisco Systems