[openib-general] CMA: port 2 loopback problems

[Michael S. Tsirkin]

Sean, I am seeing the following problem: I have a dual-port HCA
with IPoIB interfaces ib0 on port 1  and ib1 on port 2.
port 1 is down and port 2 is up,
and I try creating a connection to the loopback address 127.0.0.1.

The problem I am seeing is that I am getting RDMA_CM_EVENT_ROUTE_ERROR.
Apparently CMA attempts address resolution through port 1, which fails.

1. A quick way to solve this would be to change addr_resolve_local to
only look at devices which are UP. This is slightly ugly however,
since the device could only be momentarily DOWN.

2. A nicer, but harder, way would be to find *all* appropriate devices, and
start address resolution through all of them at the same time.  The first one to
get resolved will return to the user, others should be cancelled.

3. Finally, it is possible to cause CMA to avoid performing SA queries
for access to a local device.

I think the second approach is clearly the best thing to do, but it is also
quite nontrivial. Sean, will you be interested in a patch implementing the
first approach as a stopgap measure?

-- 
MST