[ofa-general] How to Interpret MTU reported by "ibv_devinfo" vs "ifconfig ib0"

[Ralph Campbell]

The MTU reported by "ifconfig ib0" is the MTU used by the Linux
TCP/IP network stack. The MTU reported by ibv_devinfo is the
MTU that the hardware is capable of sending. This is limited
to 4K by the Infiniband specification. The reason the network
stack can have a higher MTU is that ib_ipoib is using the RC
QP protocol to send IP messages larger than the hardware MTU.
If you use "datagram" mode for ib_ipoib, you will see that
the network stack MTU is limited to the hardware MTU - 4.

On Mon, 2008-09-15 at 11:43 -0400, Kumar, Amit H. wrote:
> Hello Everyone,
> 
> Why do we see a difference in the MTU reported here by: ibv_devinfo and "ifconfig ib0" How do we interpret them?
> 
> 
> Also Is there a document where I can read in detail about IPoIB and applications that benefit from them.
> In general I understand that Socket based applications can make use of IPoIB for a better bandwidth, thought NOT for a better transport latency.
> In short I am trying to understand the difference and advantage, for an Application using "Ethernet NIC" vs "InfiniBand HCA(IPoIB enabled)", apart from knowing that there is no advantage in terms of transport latency.
> 
> 
> Thank you!,
> Amit
> 
> # ibv_devinfo
> hca_id: mthca0
>         fw_ver:                         1.2.0
>         node_guid:                      0006:6a00:9800:e8e7
>         sys_image_guid:                 0006:6a00:9800:e8e7
>         vendor_id:                      0x066a
>         vendor_part_id:                 25204
>         hw_ver:                         0xA0
>         board_id:                       MT_0230000001
>         phys_port_cnt:                  1
>                 port:   1
>                         state:                  PORT_ACTIVE (4)
>                         max_mtu:                2048 (4)
>                         active_mtu:             2048 (4)
>                         sm_lid:                 2
>                         port_lid:               29
>                         port_lmc:               0x00
> 
> 
> 
> # ifconfig ib0
> ib0       Link encap:InfiniBand  HWaddr 80:00:04:04:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
>           inet addr:172.25.44.250  Bcast:172.25.44.255  Mask:255.255.255.0
>           inet6 addr: fe80::206:6a00:a000:e8e7/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:65520  Metric:1
>           RX packets:138863 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:195994 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:256
>           RX bytes:11000200 (10.4 MiB)  TX bytes:4321658633 (4.0 GiB)
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general