[ewg] IPoIB to Ethernet routing performance

[Ali Ayoub]

On Sun, Dec 26, 2010 at 2:57 AM, Richard Croucher
<richard at informatix-sol.com> wrote:
> The vNIC driver only works when you have Ethernet/InfiniBand hardware
> gateways in your environment.   It is useful when you have external hosts to
> communicate with which do not have direct InfiniBand connectivity.
> IPoIB is still heavily used in these environments to provide TCP/IP
> connectivity within the InfiniBand fabric.

Once you have BridgeX HW, Mellanox vNic (EoIB) driver provides IB to
EN connectivity, as well as IB to IB connectivity.
Note that IB to IB connectivity doesn't involve the bridge HW
(peer-to-peer communication) so any packet sent internally within the
IB fabric doesn't reach the bridge HW.
Today, EoIB requires the BridgeX HW, in the future, Mellanox may
support "bridge-less" mode where it can work without the bridge HW.

> The primary Use Case for vNICs is probably for virtualization servers, so
> that individual Guests can be presented with a virtual Ethernet NIC and do
> not lead to load any InfiniBand drivers.  Only the hypervisor needs to have
> the InfiniBand software stack loaded.

EoIB primary use is not virtualization, although it can support it in
better ways than other ULPs.
FYI, today running full/para virtualized driver in the Guest OS is
needed also for IPoIB.
Only when platform-virtualization solution is available, the GOS will
run IB stack (for any ULP).

> -----Original Message-----
> From: ewg-bounces at lists.openfabrics.org
> [mailto:ewg-bounces at lists.openfabrics.org] On Behalf Of Ali Ayoub
> Sent: 26 December 2010 07:43
> To: Christoph Lameter
> Cc: linux-rdma; sebastien dugue; Richard Croucher; OF EWG
> Subject: Re: [ewg] IPoIB to Ethernet routing performance
>
> On Thu, Dec 9, 2010 at 3:46 PM, Christoph Lameter <cl at linux.com> wrote:
>> On Mon, 6 Dec 2010, sebastien dugue wrote:
>>
>>> > The Mellanox BridgeX looks a better hardware solution with 12x 10Ge
>>> > ports but when I tested this they could only provide vNIC
>>> > functionality and would not commit to adding IPoIB gateway on their
>>> > roadmap.
>>>
>>>   Right, we did some evaluation on it and this was really a show stopper.
>>
>> Did the same thing here came to the same conclusions.
>
> May I ask why do you need IPoIB when you have EoIB (vNic driver)?
> Why it's a show stopper?
> _______________________________________________
> ewg mailing list
> ewg at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
>
>