[ofa-general] Enabling IP_CM warns about multicast packet drops

[Sumeet Lahorani]

Olga, Or,

Thanks for the pointers.

Does this packet drop always occur at the host or could it also occur in 
the switches (Voltaire ISR 9024)?

Also, besides the "packet len too long ..." message, is the "dropped" 
statistic in ifconfig ib0 a good way to find out if such packet drops 
are happening?

- Sumeet

Or Gerlitz wrote:
> Sumeet Lahorani wrote:
>   
>> When we enable IB connected mode and increase MTU to 65520, we see the following
>> kernel: ib0: enabling connected mode will cause multicast packet drops
>> kernel: ib0: mtu > 2044 will cause multicast packet drops.
>>     
>
>   
>> Should we not be doing this? What kind of multicast packets will be dropped?
>> If we are not using multicast, do any drivers (bonding, ipoib etc) internally use 
>> multicast in a way that will cause them to not work correctly in connected mode? 
>>     
>
> Connected mode is supported only for unicast traffic where multicast traffic keeps going over the IB UD QP whose MTU is much lower (e.g 2-4K). To close the gap between the MTU seen by the network stack to the MTU used by the UD QP, IPoIB emulates receiving an icmp packet that tells the os stack to use a different path mtu for this multicast neighbour, see
>
> ipoib_start_xmit --> 
>   ipoib_send --> 
>    ipoib_cm_skb_too_long(mcast_mtu) --> 
>     skb->dst->ops->update_pmtu(skb->dst, mtu)
>
> When IP multicast is not used, multicast is used by the network stack and bonding just for the sake of sending ARPs on the broadcast group, and IGMP where the size of both is way below the IB mtu.
>
> Or.
>