[openib-general] [PATCH] [RFC] dapltest change for iwarp

[Steve Wise]

On Fri, 2006-04-28 at 12:00 -0700, Sean Hefty wrote:
> Steve Wise wrote:
> > The Chelsio RNIC has this issue.  If the server sends the first FPDU
> > _before_ the client driver has moved the connection/qp into RDMA mode,
> > then the data is placed as streaming data and the connection must be
> > terminated (dapltest 6 exposes this intermittently).  Ammasso doesn't
> > have this issue, but other RNIC's probably will.  
> > 
> > One thing I'm experimenting with is to delay the ESTABLISHED event on
> > the server side until the first FPDU is received.  However, we still
> > probably need a way for an application to know whether the client has to
> > send first (or the server as you pointed out).
> > 
> > I believe rnic-pi has an attribute that indicated this behavior...
> 
> Can this be hidden from the user?  If the client side needs to send the first 
> message, couldn't the connection protocol send a 0-byte message that the server 
> strips out?  The ESTABLISHED event would be delayed until this message is received.
> 

Yea I guess.  A zero-length RDMA Write could be sent by the MPA
initiator after it moves the QP/connection into RDMA mode.  This will
only work if the driver has a way to know if these sorts of FDPUs arrive
at the rnic.  For Chelsio, the host driver gets no notification of
successful zero-length reads or writes.  And the host driver needs to
generate the ESTABLISHED event.  The only thing the Chelsio driver can
trigger on is a RECV completion notification.  

> Also, wouldn't this issue need to be resolved based on both the local and remote 
> device capabilities?  I.e. if the server is using an Ammasso RNIC, it needs to 
> know the behavior of the remote RNIC, in case it's Chelsio.  Is this information 
> carried as part of the connection, or does iWarp force the client to initiate 
> the first send?
> 

The MPA layer of the iWARP stack forces this by requiring that the MPA
responder (the server) not send an FPDU until it first receives one from
the client. 

Stevo.