[ewg] bug 1918 - openmpi broken due to rdma-cm changes

[Sean Hefty]

>Well then the rdma-cm needs to know which devices support hw loopback.
>Cuz on a T3-only system, no hwloop...

The problem sounds like it's more than just whether 127.0.0.1 is usable.  That
check may fix openmpi, but it sounds more like the app needs to know whether the
device can actually support loopback, regardless of what addresses are used.  Is
this correct?

What would openmpi do if there were two addresses assigned to the T3 device?
Does openmpi simply bypass RDMA for all connections on the local machine?

Basically, I'm not sure that this is *just* an rdma_cm issue.  Although it
definitely appears that some sort of change needs to be made to the rdma_cm.

- Sean