[openib-general] rdma_cm branch

[Tom Tucker]

On 10/4/06 12:40 PM, "Steve Wise" <swise at opengridcomputing.com> wrote:

> On Wed, 2006-10-04 at 10:17 -0700, Sean Hefty wrote:
>>> Guys, I must be confused.  I thought the private data _was_ passed up in
>>> the ESTABLISHED event on the active side.  We have tools in the
>>> perftools directory that utilize this.   What am I missing here?
>> 
>> When a user calls rdma_connect(), they specific connection parameters (like
>> responder_resources and initiator_depth) through a struct rdma_conn_param.
>> These parameters are NOT given to the user when the connect request event is
>> reported.
>> 
>> The issue is: are these values needed by the user during connection
>> establishment?  If yes, then how do we export them to the user.
>> 
> 
> I understand now.
> 
> For iWARP, the key parameter is setting your local QP's ORD (initiator
> resources) to <= your peer's IRD (responder resources) to avoid
> overflowing the peers incoming rdma read queue.  I think the iWARP
> devices must support setting ORD even after the connection is setup and
> the QP is in RTS, so the connection _could_ be setup (qp moved to RTS)
> and then the QP modified to the appropriate settings after querying to
> get the peer's params.  But I think it seems more natural to deal with
> this at connection setup time.
> 
> It would be nice, IMO, for the RDMA CM to handle this under the covers
> and setup the QP appropriately.  Thus the parameters need not be passed
> to the consumer...
>

Actually, I think how the IRD/ORD parameters are exchanged and negotiated by
the CM in private data is a separate issue from whether or not the end
result of the negotiation is provided to the app in the established event.

I think some apps would like to know the end result of the negotiation so
that it can throttle RDMA_READ submissions on the SQ and avoid stalling
outbound RDMA_WRITE/RDMA_SEND behind the last RMDA_READ.

So I guess that's a long way of saying that I advocate adding the negotiated
value/values to the event.

> My 2 cents.
> 
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>