[ewg] RE: [ofa-general] Re: [OMPI devel] OMPI over ofed udapl - bugs opened

[Caitlin Bestler]

> 
> 2) OMPI is not adhering to the iwarp protocol requirement
> that the ULP,
> in this case OMPI, initiating the iwarp connection (the side
> issuing the
> dat_ep_connect() or rdma_connect()) _MUST_ be the first to
> send an RDMA
> message.  So if a OMPI process _accepts_ an rdma connection, then it
> cannot send on that connection until it receives some sort of rdma
> operation from the client process.  It appears the current OMPI
> connection setup model doesn't enforce this.
> 

This is actually an MPA requirement, and accoring to *protocol* specs
having the active side send a zero length RDMA Write should be able
to fix the problem. However there is language in the RDMAC verbs that
clearly implies that the active side must Send something, and that an
RDMA Write is insufficient.

Therefore, the only truly safe thing for an iWARP btl to do (or a
udapl btl since that is also an iWARP btl) is to have the active
layer send an MPI Layer "nop" of some kind immediately after 
establishing the connection if there is nothing else to send.