[ewg] RE: [OMPI devel] [ofa-general] Re: OMPI over ofed udapl - bugs opened

[Caitlin Bestler]

Jeff Squyres wrote:

> 
> - The other peer (the receiver of the connection) must wait
> to send its pending fragment(s) until it receives the first
> frag from the connection initiator.  This can be accomplished
> either with another flag on the OMPI module struct or perhaps
> making it part of the connection protocol (i.e., don't
> transition the endpoint to be CONNECTED until the first
> fragment is received).  Either of which can be used to queue
> up fragments on the receiver until the first fragment is
> received from the initiator.  I'd have to look in the code
> deeper, but I'm *guessing* that it might be best to use the
> already-existing state flag (i.e., checking for CONNECTED)
> because then you won't be introducing any more conditionals
> in the critical path.
> 

The transport provider has several options on ensuring that
the passive side does not put a message on the wire before
the first message is received.

What the transport layer cannot do is create the first message
from the active side. Because it will have send/recv semantics
it will complete a receive work request, which the application
layer has to post with that expectation.

this nop does not have to be visible above OMPI, but I'm pretty
sure OMPI has to generate it. That isn't exactly fair to the 
application layer, but the RDMAC verbs are water under the
bridge. Assuming OMPI wants to work with *any* iWARP RNIC
then it needs to ensure that the active side will send something
promptly in all cases.