[ewg] RE: [ofa-general] [PATCH 0/9] RDMAoE - RDMA over Ethernet

[Liran Liss]

S.B.
--Liran 


>RDMA over Ethernet (RDMAoE) allows running the IB transport protocol 
>over Ethernet, providing IB capabilities for Ethernet fabrics. The 
>packets are standard Ethernet frames with an Ethertype, an IB GRH,  
>unmodified IB transport headers and payload. HCA RDMAoE ports are no 
>different than regular IB ports from the RDMA stack perspective.

I would refer to this as IBoE, not RDMAoE.

The RDMA stack should see these ports different than regular IB HCA
ports.
There are a lot of differences that should not simply be hidden or
incorrectly
assumed: QP0, QoS, multiple paths, routing(?), no SA, etc. 

LL: the RDMA stack will see that the port has different link types.
SLs map cleanly to VLAN user priorities.

>IB subnet management and SA services are not required for RDMAoE 
>operation;

Then I would not try to emulate it at all.  As Hal mentioned in a
separate post, there are too many ways to interact with the SA that an
emulation won't cover.

LL: you need to emulate *enough* so that typical applications don't need
to worry about the link type. SA path queries is the best example.
Otherwise, every RDMA application (not necessarily a CMA app) will need
to have different code paths depending on the link type.

>Ethernet management practices are used instead. In Ethernet, nodes are 
>commonly referred to by applications by means of an IP address. RDMAoE 
>treats IP addresses that were assigned to the corresponding Ethernet 
>port as GIDs, and makes use of the IP stack to bind a destination 
>address to the corresponding netdevice (just as the CMA does today for 
>IB and iWARP) and to obtain its L2 MAC addresses.

Is the actual L3 address an IP address, or just an encoded IP address in
an IBoE
L3 address?  What L3 protocol is being used and will it interoperate
with some peer L3 protocol (IP or IB)?

LL: RDMAoE uses GIDs that encoded IP addresses. For IPv6, this is
straightforward. We use mapped address for IPv4 (::0xffff<ipv4
address>).
Currently, RDMAoE is not routable, as the IB routing specs are not
complete.
However, nothing prohibits making it so in the future (either Eth to Eth
or Eth to IB).

- Sean

_______________________________________________
general mailing list
general at lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general

To unsubscribe, please visit
http://openib.org/mailman/listinfo/openib-general