[openib-general] Connectivity

[Hal Rosenstock]

On Fri, 2005-09-02 at 08:28, Sean Hubbell wrote:
> Again, thanks Hal. Yes, I can perform an ibping on all of the nodes so 
> connectivity and the ports inbetween are up. I am almost positive now 
> that this has something to do with the IPoIB. I am going to try to ping 
> each node and then look at the arp table. Do you know of anything I can 
> do to look specifically at the IPoIB "exchange".

If ibping works, UD unicast is working (and would work for that part of
IPoIB). What I suspect is not working is multicast. I suspect some issue
with the IPoIB broadcast group. So can you comment on the topology and
provide an OpenSM log when run with verbose ? [Also can you down and
then up all the ib<n> interfaces and see if connectivity is restored.
Also, is the SM running on a node also running IPoIB ?]

If not, you can debug this using the following:

1. Using ibroute, you can display the multicast tables in the switches.
Using ibtracert you can trace the route of a multicast group.

        Multicast examples:
                ibroute -M 4    # dump all non empty mlids of switch with lid 4
                ibroute -M 4 0xc010 0xc020      # same, but with range
                ibroute -M -n 4 # simple dump format

        Multicast example:
                ibtracert -m 0xc000 4 16        # show multicast path of mlid 0xc000 between lids 4 and 16

2. There are 2 levels of debug tracing in IPoIB. You can enable these in
the build with CONFIG_INFINIBAND_IPOIB_DEBUG and
CONFIG_INFINIBAND_IPOIB_DEBUG_DATA

-- Hal