[ofa-general] post_recv question

[Jeff Squyres]

On Feb 21, 2008, at 10:19 AM, Gleb Natapov wrote:

>> To further complicate things, this race condition is never seen  
>> _if_ the
>> application uses the same QP to advertise (send a credit allowing the
>> peer to SEND) the RECV buffer availability.  So if the app posts a  
>> SEND
>> after the RECV is posted and that SEND allows the peer access to the
>> RECV buffer, then everything is ok.  This is due to the fact that the
>> FW/HW will process the SEND only after processing the RECV.  If the  
>> app
>> uses a different QP to post the SEND advertising the RECV, then the  
>> race
>> condition exists allowing the peer to SEND into that RECV buffer  
>> before
>> the HW makes it ready.
>>
> OpenMPI can be configured to send credit updates over different QP.  
> I'll
> try to stress it next week to see what happens.


FWIW: this is exactly where the question arose: Steve's working on the  
iwarp port of OMPI, and since we send the flow control messages for  
all QP's between a pair of processes over a single QP, this apparent  
race condition can occur.

-- 
Jeff Squyres
Cisco Systems