[openib-general] [PATCH] osm: PathRecord prefer 1K MTU for MT23108 devices

[Michael S. Tsirkin]

Quoting r. Or Gerlitz <ogerlitz at voltaire.com>:
> Subject: Re: [PATCH] osm: PathRecord prefer 1K MTU for MT23108 devices
> 
> Michael S. Tsirkin wrote:
> > Quoting r. Or Gerlitz <ogerlitz at voltaire.com>:
> 
> >> Eitan Zahavi wrote:
> >>> The following patch solves an issue with OpenSM preferring largest MTU 
> >>> for PathRecord/MultiPathRecord for paths going to or from MT23108 (Tavor)
> >>> devices instead of using a 1K MTU which is best for this device.
> 
> >> Isn't the 2K MTU issue with Tavor comes into play only under RC QP?
> 
> > I don't think so, no. Tavor supports 2K MTU, but it has better performance with
> > 1K MTU than 2K MTU. QP type should not matter.
> 
> Can you double check that please, as far as i know there is something 
> like BW 40-50% drop with Tavor/RC/2048 vs Tavor/RC/1024 but the BW with 
> Tavor/UD/2048 is **no less** then Tavor/UD/1024.

The property of Tavor to work better with 1K MTU is not transport-specific.
But, BW depends on the ULP. I guess UD top BW is simply lower (smaller messages)
so you do not see the drop there.

This just means ULP should use MTU selector to give SM hints about the MTU it
wants. If it wants the highest MTU available it should set the selector to 3,
not wildcard it.

> So its very common for IPoIB net devices impl. to expose 2044 or 1500 
> bytes MTU to the OS eg to cope with Ethernet and reduce IP 
> fragmentation/reassembly of UDP/TCP traffic.

I expect IPoIB to get better performance with higher MTU - TCP
fragmentation likely has bigger effect than hardware speed quirks.
But this is just another reason to set the mtu selector in IPoIB
appropriately.


-- 
MST