[ofa-general] [PATCH 0/ 8] XRC patch series (including xrc receive-only QPs)

[Jack Morgenstein]

This patch series is the updated XRC implementation
(kernel and user (libibverbs and libmlx4)).

Please give feedback -- I'm still reviewing the locking in this implementation.

The kernel patches are all based on
git: //git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git
branch: for-2.6.25
commit: 5e8a3c6041ded7e306607bb6c96a0e68ca4dd2b4

*****
In addition, the kernel patch series requires that Eli Cohen's 
patch 7/16, posted January 16, be applied first
([ofa-general] [PATCH 7/16] ib/core: Add creation flags to QPs )
*****

The patches should be applied in the order posted.

Changes:
- Added creation of XRC receive-only QPs for userspace, which
  reside in kernel space (user cannot post-to or poll these QPs).
  Motivation:  MPI community required XRC receive QPs which would
               not be destroyed when the creating process terminated.

  Solution:  Userspace requests that a QP be created in kernel space.
             Each userspace process using that QP (i.e. receiving
             packets on an XRC SRQ via the qp), registers with
             that QP (-- the creator is also registered, whether
             or not it is a user of the QP). When the last userspace user
             unregisters with the QP, it is destroyed.  Unregistration
             is also part of userspace cleanup, so there is no leakage.

  API for this:
	ibv_create_xrc_rcv_qp
	ibv_modify_xrc_rcv_qp
	ibv_query_xrc_rcv_qp
	ibv_reg_xrc_rcv_qp
	ibv_unreg_xrc_rcv_qp

  Creating process workflow:
	ibv_create_xrc_rcv_qp -- to create
	ibv_modify_xrc_rcv_qp -- to move QP to INIT
	ibv_modify_xrc_rcv_qp -- to move QP to RTR
              (to RTS is not needed for receive-only QPs)

	ibv_unreg_xrc_rcv_qp -- instead of destroy.

  Using process workflow
	ibv_create_xrc_srq -- to create an SRQ
	ibv_reg_xrc_rcv_qp -- to register with the QP as a user

	ibv_destroy_srq
        ibv_unreg_xrc_rcv_qp -- to "unregister" with the QP.  If no
				user process remain registered, the
				QP is destroyed.

NOTES:
  1. Since there is no userspace object for the QP, the API uses
     the XRC domain object and qp number instead.
  
  2. Registration needs to be performed only once per process
     (multiple registrations count as a single registration).

  3. Async events for the receive QP are delivered to all registered
     processes.  The event ID is "OR'ed" with 0x80000000, to indicate
     that this is an XRC receive-only QP event.  The element field
     union value "xrc_qp_num" is set to the QP number which generated the
     event.

- Jack