<table cellspacing="0" cellpadding="0" border="0" ><tr><td valign="top" style="font: inherit;">Dear all,<br>I am new to this field and I have some questions.<br>I run a cluster with IB Mellanox. I have two subnets each with its own opensm (running on different port P1 or P2). mixed hardware MT25204 and MT23108. all are at 4x rate.<br>mixed drivers IBGold1.8.2 and MLNX_OFED_LINUX-1.3.1-rhel4<br>when I issue <br>ibdiagnet -p 1 -lw 4x I get <br>-I- Stages Status Report:<br>    STAGE                                    Errors Warnings<br>    Bad GUIDs/LIDs Check                     0      0<br>    Link State Active
 Check                  0      0<br>    Performance Counters Report              0      0<br>    Specific Link Width Check                0      0<br>    Partitions Check                         0      0<br>    IPoIB Subnets Check                      0      16<br>BUT<br>-I---------------------------------------------------<br>-I-
 IPoIB Subnets Check<br>-I---------------------------------------------------<br>-I- Subnet: IPv4 PKey:0x7fff QKey:0x00000b1b MTU:4096Byte rate:120Gbps SL:0x00<br>-W- Port h1/P1 lid=0x0011 guid=0x dev=23108 can not join due     to rate:10Gbps < group:120Gbps<br>-W- Port h2/P1 lid=0x0321 guid=0x dev=23108 can not join    due to rate:10Gbps < group:120Gbps<br>-W- Port h3/P1 lid=0x0069 guid=0x dev=23108 can not join due to    rate:10Gbps < group:120Gbps<br>-W- Port h4/P1 lid=0x0010 guid=0x dev=23108 can not    join due to rate:10Gbps < group:120Gbps<br>and so on with all the nodes.<br>switch type MT47396.<br><br>I think the problem is  this line <br>Subnet: IPv4 PKey:0x7fff QKey:0x00000b1b MTU:4096Byte rate:120Gbps SL:0x00<br>but I don't know how to set the Mtu to 2048 and rate to 10G.<br><br>more<br> /usr/sbin/saquery -d -g<br><br>MCMemberRecord group
 dump:<br>                MGID....................0xff12401bffff0000 : 0x00000000ffffffff<br>                Mlid....................0xC000<br>                Mtu.....................0x5<br>                pkey....................0xFFFF<br>                Rate....................0xA<br>MCMemberRecord group dump:<br>                MGID....................0xff12401bffff0000 : 0x0000000000000001<br>               
 Mlid....................0xC001<br>                Mtu.....................0x5<br>                pkey....................0xFFFF<br>                Rate....................0xA<br>MCMemberRecord group dump:<br>                MGID....................0xff12401bffff0000 : 0x0000000000656565<br>                Mlid....................0xC002<br>                Mtu.....................0x4<br>               
 pkey....................0xFFFF<br>                Rate....................0x3<br>MCMemberRecord group dump:<br>                MGID....................0xff12401bffff0000 : 0x0000000000a847ff<br>                Mlid....................0xC003<br>                Mtu.....................0x4<br>                pkey....................0xFFFF<br>                Rate....................0x3<br>MCMemberRecord group dump:<br>               
 MGID....................0xff12401bffff0000 : 0x0000000000000000<br>                Mlid....................0xC007<br>                Mtu.....................0x4<br>                pkey....................0xFFFF<br>                Rate....................0x2<br>MCMemberRecord group dump:<br>                MGID....................0xff12401bffff0000 : 0x000000000202c902<br>                Mlid....................0xC008<br>               
 Mtu.....................0x4<br>                pkey....................0xFFFF<br>                Rate....................0x2<br><br>the question is how can I set the group rate to 10G and not 120G? and the group MTU to 2048 as on some nodes I get " failed to join multicast or setting  MTU>4096 will ...generate...some errors"<br><br>thank you very much!<br>Vali<br></td></tr></table><br>