[ofa-general] Re: [PATCH/RFC] IPoIB: Allocate priv->tx_ring with vmalloc()

[Roland Dreier]

 > I noticed that too and I am not sure what the solution should be. From
 > one side, for CM mode we clear NETIF_F_SG but then we use an MTU of
 > 65520 bytes and it might not be so trivial for the networking stack to
 > provide SKB with a linear data so large.

But that's the network stack's problem.  Since we clear NETIF_F_SG
(and since we don't have checksum offload for connected mode) there's
no way that we would ever get an skb with any fragments when we enable
CM.  And the current IPoIB CM code has no handling for non-linear
skbs.  So I don't see any point in worrying about this.

 > Maybe we could use a struct defined like this:
 > 
 > struct ipoib_tx_buf {
 > 	struct sk_buff *skb;
 > 	u64		mapping[0];
 > };
 > 
 > and when allocating, allocate as much memory as needed to cover the
 > required size of the "mapping" array. We'll have to use kmllaoc n times
 > according to the size of the tx ring. This could be good also for 32 bit
 > systems where vmalloc area is small.

We could do this for the main (datagram mode) tx_ring, but then we eat
an extra pointer indirection for every send.  I suspect that's worse
than the TLB miss we take by using vmalloc().

 - R.