[ofa-general] [UPDATE][PATCH 0/3] ib/ipoib: Enable IPoIB-UD 4K MTU support (for-2.6.25)

[Shirley Ma]

Hello Roland,

Finally I cleaned up my local git tree and built compile this patchset
against 2.6.25 kernel. The patchset has splitted into three patches, the
patch could be built in sequence separately so it's easy to test.

1/3. Make IPoIB-CM RX S/G APIs generic
2/3. Set IPoIB-UD RX S/G parameters
3/3. Enable IPoIB-UD RX S/G

Please review these patches as soon as possible so we can meet 
OFED-1.3-RC4 schedule.

Appreciate your help on time.

The current IPoIB-UD implementation is limited IPoIB payload size to
2048 through hard coding IPOIB_PACKET_SIZE. The implementation is
designed for kernel PAGE_SIZE equals or greater than 4K. If the 
kernel PAGE_SIZE is equals to 2K, memory buffer allocation will be failed 
when lack of large buffer of memory. However most of the Distros 
does support PAGE_SIZE >= 4K. So this implementation has no problem 
for 2048 payload.This implementation is simple but it prevents HCA 
device who does support 4096 payload from performing, like IBM eHCA2.

This patch allows IPoIB-UD MTU up to 4092 (4K - IPOIB_ENCAP_LEN) when
HCA can support 4K MTU. In this patch, APIs for S/G buffer allocation in
IPoIB-CM mode has been made generic so IPoIB-UD and IPoIB-CM can share
the S/G code. When PAGE_SIZE is equal or greater than IPOIB_UD_BUF_SIZE
+ bytes padding to align IP header, Only one buffer is needed for 4K MTU
buffer allocation, otherwise, two buffers allocation is needed in S/G.

The node IPoIB link MTU size is the minimum value of admin configurable
MTU through ifconfig and IPoIB default broadcast group MTU size. When
Subnet Manager enables default broadcast group during start up, this
subnet IPoIB link MTU will be the value of default broadcast group MTU
size. For any node IB MTU smaller than this value, the node can't join
this IPoIB subnet. For any node IB MTU is greater than this value, the
node will join this IPoIB subnet and this value will be set as its IPOIB
link MTU. If Subnet Manager disables default broadcast group during
start up, the first bring up node in this subnet will create the default
IPoIB broadcast group based on the negotiation with the Subnet Manager,
the default is currently set as 2K according to IPoIB RFC.

Thanks
Shirley