[rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support

[Olaf Kirch]

Oops. There's an even bigger problem I didn't notice until now.

You pass the rds socket into get_mr in order to get a rds_conn
and the QP. Because you need a QP to post your fastreg WR to.

However, that doesn't work the way you approach it.

The rs_conn inside the rds_socket is just a cached pointer.
It's what we got when we called rds_conn_lookup in the
most recent call to rds_sendmsg - we do this for efficiency
reasons, because it helps us avoid taking a spinlock.

So in other words, rds_sock->rs_conn may be NULL, it may
point to some random unrelated rds_connection - and in some
cases it may even point to the rds_conn you're interested
in.

We have two choices here

 -	change the GET_MR API so that the caller includes
	the destination IP - then we can always locate the
	correct conn.

 -	require that iWARP RDMA only works with the new
	cmsg based interface; in this case the MR mapping
	happens from inside sendmsg, where we naturally
	have the destination address

Neither of these will make the Oracle IPC layer happy.
AFAIK Oracle IPC doesn't pass around the destination IP
when allocating a MR using GET_MR. And it doesn't support
the new get_mr interface either.

To what degree are we bound to a specific QP? In other words,
if I want to do RDMA over a given QP, do I have to perform
the fastreg over this specific QP, or could I use any QP?
So, would it work to create a QP specifically for the purpose
of performing fastregs?

If a fastreg is bound to a specific QP, then what happens if
the RDS connection bounces and we have to allocate a fresh
QP? Would that render all existing fastregs invalid?

Olaf
-- 
Olaf Kirch  |  --- o --- Nous sommes du soleil we love when we play
okir at lst.de |    / | \   sol.dhoop.naytheet.ah kin.ir.samse.qurax