[openib-general] How about ib_send_page() ?

[Vivek Kashyap]

On Tue, 24 May 2005, Roland Dreier wrote:

>     Vivek> I should say it depends. One can utilise a setup that sets
>     Vivek> the MTU to 2044 or whatever is the UD MTU on the subnet for
>     Vivek> all modes. The connection will advertise the maximum
>     Vivek> receive MTU as this value.
> 
> OK, but that's throwing away the main advantage of connected mode
> IPoIB, right?

I was addressing the case of a minimal implementation. For sucn an an 
initial implementation one can go as above. However, as I noted 
in the previous mail, if a large MTU is to be supported then one has to
split the CM and UD MTUs since the UD MTU is limited to 2 or 4K.
It is the limitation of the media that we get large MTU with connected mode
whereas we get multicast (absolutely necessary for for address resolution)
only with UD unless we want to do multicast emulation.


> 
> For full connected mode support, do you see an alternative to handling
> per-destination MTUs?

..how about the following:

To send a node  must be aware of the remote node's maximum MTU. The
the  node can determine the minimum of these maximums. 
Therefore, one can go with a two way split -- for UD use the UD MTU. For any
RC communication use this minimum (or less depending on the packet size). 
Therfore, one can decrement the RC MTU if needed at every new RC connection
that is made - i.e. one does recognise the receive MTU from the peer but 
uses it to decrement the overall RC MTU if required. It will not be optimal 
for all cases since some could do with larger MTU  but will interoperate. One 
can also administratively set a large enough 'minimum RC MTU' on all the nodes 
in the subnet.

Vivek

> 
>  - R.
> 
>