[ofa-general] RFCv2: SRC API

[Tang, Changqing]

> +When job j1 needs to transmit to job j2 on remote node n for 
> the first time:
> +1. Test: does job j1 have an existing connection to some job 
> on node n?
> +        - If no:
> +		j1 creates an SRC QP qp1 (send QP)
> +			qp1 is only used to post send WRs
> +		j2 creates an SRC QP qp2
> +			qp2 is part of SRC domain
> +			qp2 is only used to do transport checks:
> +				neither send nor receive WRs 
> are posted on qp2
> +		j1 and j2 create a connection between qp1 and qp2
> +	- If yes:
> +		let qp1 be the QP which belongs to j1 and is connected
> +			to some qp on node n
> +
> +2. j1 gets SRQ number from j2
> +3. j1 can now use QP qp2 from step 1
> +   and SRQ number from step 3 to send data to j2
> +
> +Cleanup:
> +When job j1 does not need to communicate to any jobs on node n, it 
> +disconnects qp1 from qp2, and asks j2 to destroy qp2.

Suppose remote node n has j2/qp2, j3/qp3, j4/qp4, qp1 on j1 is connected
to qp2, then there is no need to make connection between qp1 and qp3,
qp1 and qp4, there are automatically connected, right ?  Then how can j3
know that j2 has connected to j1, it does not need to make connection
again ?

qp1 find destination by SRQ number only, so "+3. j1 can now use QP qp2
from step 1", what does it mean ?

Can we destroy qp2 on j2 first, and keep j1 and j3 continue to
communicate ?

--CQ