[ofa-general] IBCM and ordering of OpenFabrics devices

[Hefty, Sean]

>1. Technically, mlx4_0 is the first IB device. Should it therefore be
>using ucm0? I.e., is libibcm wrong for trying to use ucm1? (note that
>OMPI's openib BTL is currently replicating the logic from libibcm to
>check for the Right ucm* file so that we can silently fail before
>ib_cm_open_device() fails with a warning message -- so if libibcm's
>logic to find the Right ucm* file changes, we'll also need to change
>MPI's logic to mirror it. OMPI's logic becomes moot in newer libibcm
>versions where Sean removed the warning message, though).

I think what may be happening is that uverbs1 should map to ucm0, but
the library opens ucm1 instead.  And uverbs2 should map to ucm1, but an
attempt is made to open ucm2, which causes the failure that you're
seeing.  The libibcm needs to determine the correct file to open using
something other than just the uverbs name.

That was just a convenient way to get the correct file name, but was
never updated once libibverbs supported non-IB devices.

- Sean