[openib-general] RFC: CMA backlog

[Michael S. Tsirkin]

Hello, Sean!
I am looking at implementing the listen backlog parameter correctly.
Here's what this does in TCP: TCP counts the number of connect requests
at the specific local socket that were not yet accepted by accept().
Once this number exceeds the backlog specified in listen, new SYN
packets will be dropped, and the remote side will retry.

Implementing this correctly is important for scalability when a lot of clients
connect to a single server - this way if there is a pressure on server to handle
connections, it is distributed between clients who have to retry.

Currently, this is not something that can be implemented by ULP on top of
CMA, because returning error from REQ will result in reject rather than REQ
drop.

CMA already has backlog parameter in listen but it is ignored as far as I can
see. I propose extending cma API with the following options:

rdma_backlog_added - connection was added to backlog queue
rdma_backlog_removed - connection was removed from backlog queue

Internally, CMA will count the # of connections in backlog. If
If REQ arrives and this number exceeds the backlog given in listen,
CMA will drop the REQ, without creating the new CMA ID.

This mimics the TCP behaviour closely an will hopefully scale well.

Opinions?

-- 
MST