[ewg] [PATCH 0/9] RDMAoE - RDMA over Ethernet -- some procedural questions

[Ryan, Jim]

I'm responding to this posting because it offered a clear summary of the RDMAoE proposal from Mellanox.

I want to raise some possible procedural questions and ask for comment. I don't have detailed notes on this, so this is to the best of my recollection. I recall RDMAoE was introduced, I think for the first time, at an EWG meeting. There were some concerns raised then and the Mellanox proponents withdrew the proposal. The concerns raised included:
- There was no documentation on what Mellanox was proposing

- There was no source code

- There were concerns about potential IP issues

- There was no standards body associated with the proposal. 

Recall the bylaws of OFA requires that any ULPs that OFA supports will be produced by some recognized standards organization. No such organization was known to be associated with the Mellanox proposal.

So, the simple question is do we have an issue or is the Mellanox proposal somehow exempt from our bylaws requirements?

Thanks, Jim 

-----Original Message-----
From: ewg-bounces at lists.openfabrics.org [mailto:ewg-bounces at lists.openfabrics.org] On Behalf Of Eli Cohen
Sent: Monday, June 15, 2009 12:34 AM
To: Roland Dreier
Cc: ewg; general-list
Subject: [ewg] [PATCH 0/9] RDMAoE - RDMA over Ethernet

RDMA over Ethernet (RDMAoE) allows running the IB transport protocol over
Ethernet, providing IB capabilities for Ethernet fabrics. The packets are
standard Ethernet frames with an Ethertype, an IB GRH,  unmodified IB transport
headers and payload. HCA RDMAoE ports are no different than regular IB ports
from the RDMA stack perspective.
    
IB subnet management and SA services are not required for RDMAoE operation;
Ethernet management practices are used instead. In Ethernet, nodes are commonly
referred to by applications by means of an IP address. RDMAoE treats IP
addresses that were assigned to the corresponding Ethernet port as GIDs, and
makes use of the IP stack to bind a destination address to the corresponding
netdevice (just as the CMA does today for IB and iWARP) and to obtain its L2
MAC addresses.
    
The RDMA Verbs API is syntactically unmodified. When referring to RDMAoE ports,
Address handles are required to contain GIDs and the L2 address fields in the
API are ignored. The Ethernet L2 information is then obtained by the
vendor-specific driver (both in kernel- and user-space) while modifying QPs to
RTR and creating address handles.
    
In order to maintain application compatibility, RDMAoE implements a SA_Query
API that locally returns path records with the corresponding GIDs and the other
relevant parameters . Consequently, any CMA or native Verbs application, in
kernel or user-space, that uses path queries to obtain its address information,
will run transparently over RDMAoE with no changes. We have successfully tested
MPI, SDP, RDS, and native Verbs applications over RDMAoE without *any* changes.
    
In the mlx4 driver stack, mlx4_en must be loaded and the corresponding eth
Ethernet (with support for RDMAoE) or IB, as it was already the case.

Following is a series of 9 patches based on version 2.6.30 of the
Linux kernel.

Signed-off-by: Eli Cohen <eli at mellanox.co.il>
    

 drivers/infiniband/core/addr.c            |   20 ++-
 drivers/infiniband/core/agent.c           |   16 +-
 drivers/infiniband/core/cma.c             |   39 ++++-
 drivers/infiniband/core/mad.c             |   48 ++++--
 drivers/infiniband/core/multicast.c       |  153 ++++++++++++++--
 drivers/infiniband/core/sa_query.c        |  167 ++++++++++++++----
 drivers/infiniband/core/ud_header.c       |  111 ++++++++++++
 drivers/infiniband/core/uverbs.h          |    1 +
 drivers/infiniband/core/uverbs_cmd.c      |   34 ++++
 drivers/infiniband/core/uverbs_main.c     |    1 +
 drivers/infiniband/core/verbs.c           |   18 ++
 drivers/infiniband/hw/mlx4/ah.c           |  228 ++++++++++++++++++++----
 drivers/infiniband/hw/mlx4/main.c         |  276 ++++++++++++++++++++++++++---
 drivers/infiniband/hw/mlx4/mlx4_ib.h      |   30 +++-
 drivers/infiniband/hw/mlx4/qp.c           |  253 +++++++++++++++++++++-----
 drivers/infiniband/ulp/ipoib/ipoib_main.c |    3 +
 drivers/net/mlx4/cmd.c                    |    6 +
 drivers/net/mlx4/en_main.c                |   15 ++-
 drivers/net/mlx4/en_port.c                |    4 +-
 drivers/net/mlx4/en_port.h                |    3 +-
 drivers/net/mlx4/intf.c                   |   20 ++
 drivers/net/mlx4/main.c                   |    6 +
 drivers/net/mlx4/mlx4.h                   |    1 +
 include/linux/mlx4/cmd.h                  |    1 +
 include/linux/mlx4/device.h               |   31 +++-
 include/linux/mlx4/driver.h               |   16 ++-
 include/linux/mlx4/qp.h                   |    8 +-
 include/rdma/ib_addr.h                    |   51 ++++++
 include/rdma/ib_pack.h                    |   26 +++
 include/rdma/ib_user_verbs.h              |   21 ++-
 include/rdma/ib_verbs.h                   |   22 +++
 31 files changed, 1419 insertions(+), 210 deletions(-)
_______________________________________________
ewg mailing list
ewg at lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg