[ofa-general] impossibility to bind a device/port with the rdma-cm when the port is down

[Or Gerlitz]

Hi Sean,

It turns out that with the IPOIB port-space, its impossible to bind a device/port
through the rdma-cm when the port is down. This is as of the following call sequence
cma_acquire_dev --> cma_set_qkey/ps=IPOIB --> ib_sa_get_mcmember_rec where the
latter returns EADDRNOTAVAIL since when the port went down the core multicast code
flushed its data base. I see that the qkey is actually used by the rdma-cm when
the user attepts to join a multicast group, when "connect" a UD QP and when creating
a UD QP. I  assume there must be a way to defer this resolving to a later stage such
that binding would be possible when the port is down, thoughts?

Or.

$ udaddy -b 10.10.5.157  -p 2
udaddy: starting server
udaddy: bind address failed: -1
test complete
return status -1

$ strace udaddy -b 10.10.5.157  -p 2
[...]
write(5, "\2\0\0\0(\0\0\0\0\0\0\0\0\0\0\0\2\0\6\34\n\n\5\235\0\0\0\0\0\0\0\0"..., 48) = -1 EADDRNOTAVAIL (Cannot assign requested address)
write(1, "udaddy: bind address failed: -1\n", 32udaddy: bind address failed: -1) = 32