[ofa-general] [PATCH] IPoIB UD 4K MTU support

[Shirley Ma]

Hello Eli,

> I don't think it's a good idea to make the code more generic and use the
> same rx buffer scheme for both UD and CM. In the case of UD we would
> need 2 scatter entries - on for the dummy GRH bytes which can be
> initialized once to point at the same buffer, and the second to point to
> the real data buffer. I prefer to modify the UD code to work with 4K
> MTU.
> The reasoning is that CM scatters can reach on overall size of 17 which
> would require more memory and would consume more CPU cycles to handle
> (e.g. when a packet is received). This can be crucial for the cases
> where small UDP packets performance is needed.

What I meant here is to have some generic APIs for buffer allocation
based on PAGE_SIZE, IPoIB payload size, HEAD_SIZE. In this way, if
PAGE_SIZE is greater than IPoIB payload size (IPoIB-CM payload size is
different than IPoIB-CM size) plus padding (IPoIB-CM padding is
different than IPoIB-UD padding) to align IP header to 16 bytes, then
only one buffer is needed. Number of buffers is dependent on IPoIB
payload size + padding size. For example, let's assume the PAGE_SIZE =
4K, only one buffer is allocated for 2K MTU (IPoIB mtu is 2K-4), two
buffer are allocated for 4K MTU(IPoIB mtu is 4K-4), 16 buffers are
allocated for 64K MTU(IPoIB mtu is 64K-12). If PAGE_SIZE = 64K, then
only one buffer is allocated for both IPoIB-UD and IPoIB-CM.

Actually, I found a problem in IPoIB-CM here. Looks like most of HCAs do
support S/G numbers > 16. Then the other OS implementation could have
IPoIB link MTU = 64K -4 instead of current implementation 64K-16. It
could be a problem for this implementation to be in a mixed OS
evviornment. Just like my previous simple approach implementation for
IPoIB-UD, which is IPoIB link MTU = 4K - 48.

Roland,

	Do you want me to fix this?

Thanks
Shirley