[openib-general] Re: CMA stuff

[Caitlin Bestler]

> 
>>> 	- It seems that in ucma backlog is checked when a connection
>>> 	  request arrives. However this is not how TCP handles backlog,
>>> 	  so socket apps being ported to CMA might hit a problem.
>> ...
>> I think that I need a specific example (i.e. a patch) to see how you
>> would treat backlog differently.
> 
> I'm still trying to get my head about the cma implementation, but
> I'll see what I can do. 
> 
> My idea was to have 2 counters: one counting connections that
> are being established, gong up to tcp_max_syn_backlog, and
> another one after the connection is established, up to local
> socket backlog.

Whether it is over IB or iWARP, the caller needs to be able
to protect itself from excessive callbacks. This can be done
in advance, with a backlog parameter, or by allowing it to
refuse a callback (I'm busy, just hang-up on this caller).

The backlog parameter is traditional, and it also provides
guidance to the lower layer on optimizing its resources.

Trying to get more specific will get very complex.
A "pending connection" can be in many intermim states.
For iWARP those include: SYN received (not even knowable
if SYN Cookies are in use), Connection established by
MPA Request Frame not yet received, MPA Request Frame
received and Connection Request generated but not
yet delivered to the consumer, and delivered to consumer
but not yet acknowledged. I'm sure that mapping the
backlog to "IB" would have similar issues.

The point is that you cannot define the backlog in a 
way that makes sense for *all* iWARP devices (because
it depends on where the connections are established
and whether SYN Cookies are used), let alone a model
that makes sense for both iWARP and IB.

The only sensible meaning for "backlog" is that it
is a credit given by the application to the CMA. It
is permission to make N callbacks with connection
requests. Each callback decrements the credit.
Each accept/reject restores a credit. This is
totally analogous to having a single credit
for how many in-flight requests a server allows
a client to send, without any knowledge of how
the client is structured internally or how that
limit impacts the clients decisions.