[ofa-general] can not join due to rate:2.5Gbps < group:10Gbps?

[Hal Rosenstock]

On Fri, 2008-04-04 at 10:36 -0400, Brian J. Murrell wrote:
> I'm trying to get a few nodes here connected with IPoIB.  On the first
> node I have tried with, after ifconfig'ing the interface into the
> network with other IPoIB nodes I cannot seem to ping any other nodes.  I
> ran ibdiagnet and got a /tmp/ibdiagnet.pkey file with the following
> contents:
> 
> sata14:/ # cat /tmp/ibdiagnet.pkey
> GROUP PKey:0x7fff Hosts:4
>    Full sata15/P2 lid=0x0004 guid=0x00066a01a0000363 dev=23108
>    Full sata14/P2 lid=0x0006 guid=0x00066a01a00002bf dev=23108
>    Full sata23/P2 lid=0x0008 guid=0x00066a01a00002fe dev=23108
>    Full sata16/P2 lid=0x0007 guid=0x00066a01a00002c1 dev=23108
> 
> When I run an "ibdiagpath -l 0x0004" I get the following:
> 
> -W- Topology file is not specified.
>     Reports regarding cluster links will use direct routes.
> -I- Using port 2 as the local port.
> 
> -I---------------------------------------------------
> -I- Traversing the path from local to destination
> -I---------------------------------------------------
> -I- From: lid=0x0006 guid=0x00066a01a00002bf dev=23108 sata14/P2
> -I- To:   lid=0x0001 guid=0x00066a00c8000180 dev=5 Port=1
>     
> -I- From: lid=0x0001 guid=0x00066a00c8000180 dev=5 Port=2
> -I- To:   lid=0x0004 guid=0x00066a01a0000363 dev=23108 sata15/P2
>     
> 
> -I---------------------------------------------------
> -I- PM Counters Info
> -I---------------------------------------------------
> -I- No illegal PM counters values were found
> 
> -I---------------------------------------------------
> -I- Path Partitions Report
> -I---------------------------------------------------
> -I- Source sata14/P2 lid=0x0006 guid=0x00066a01a00002bf dev=23108 Port 2
>     PKeys:0xffff
> -I- Destination sata15 lid=0x0004 guid=0x00066a01a0000363 dev=23108 PKeys:0xffff
> -I- Path shared PKeys: 0xffff
> 
> -I---------------------------------------------------
> -I- IPoIB Path Check
> -I---------------------------------------------------
> -I- Subnet: IPv4 PKey:0x7fff QKey:0x00000000 MTU:2048Byte rate:10Gbps SL:0x00
> -W- Port sata14/P2 lid=0x0006 guid=0x00066a01a00002bf dev=23108 can not join due
>     to rate:2.5Gbps < group:10Gbps
> -W- Port sata15/P2 lid=0x0004 guid=0x00066a01a0000363 dev=23108 can not join due
>     to rate:2.5Gbps < group:10Gbps
> -E- No IPoIB Subnets found on Path! Nodes can not communicate via IPoIB!
> 
> -I---------------------------------------------------
> -I- QoS on Path Check
> -I---------------------------------------------------
> -W- Blocked VLs:4 5 at node:sata14 lid=0x0006 guid=0x00066a01a00002bf dev=23108
>     port:2
> -W- Blocked VLs:4 5 at node: lid=0x0001 guid=0x00066a00c8000180 dev=5 port:2
> -I- The following SLs can be used:0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
>  
> -I- Done. Run time was 0 seconds.
> 
> That IPoIB Path Check looks a bit alarming.
> 
> Anyone have any suggestions?

Looks like you have a mixed rate set of ports so you need to configure
the group to 2.5 Gbps. What SM are you using ?

-- Hal

> b.
> 
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general