[openib-general] OpenIB and OpenRDMA: Convergence on common RDMA APIs and ULPs for Linux

[Venkata Jagana]

I am sure the developers in both of these communities have strong opinions
in one way or
the other about the use of common interface and whatever it is but
ultimately we need to
find a best possible way to move forward in order to support IB, RNICs and
other future
RDMA fabrics. :)

Absolutely, we would like to see a common mid layer with RDMA-specific
components
(for example, IB-CM and RNIC-CM would mostly be different but with very
minimal commonality
between them) developed but also should work to provide a transport-neutral
interface.

Today, we do not have a RDMA transport-neutral support with IB verbs PI in
the kernel but
 we would like to see such an interface so that we can extend it to support
variety of RDMA fabrics.
Just because we have some interface supported in the kernel today doesn't
mean that
we need to live with that interface forever. Within Linux kernel, we
absolutely want to develop
an interface that is extendable to support the future fabrics and we can
evolve this through
an open source process that is acceptable to linux kernel community.

And BTW, I am not suggesting to rip of this existing code right away but
rather continue to use and
evolve it.

I would like to really understand the technical reasons why you say
RNIC-PI is irrelevant to Linux kernel.
RNIC-PI is developed to support not only the RNICs  but it is also IB
compatible. This interface is
something that is developed just like the  open source process but with
much broader community effort
and also with review of linux OpenRDMA community and kernel community
members.

Thanks
Venkat

Roland Dreier <roland at topspin.com> wrote on 05/25/2005 07:35:52 PM:

> I believe the way forward is to evolve the existing drivers/infiniband
> code already in Linux into a drivers/rdma that supports both IB and
> RNICs.  To be extremely blunt, I believe the RNIC-PI is irrelevant to
> the Linux kernel -- no IB vendors will support ripping out a working
> midlayer and starting from scratch, and it doesn't make sense to have
> two essentially equivalent midlayers in the same kernel.
>
> To put a really concrete proposal on the table, I would suggest to
> start by extending the current ib_client registration structure, which
> looks like
>
>    struct ib_client {
>       char  *name;
>       void (*add)   (struct ib_device *);
>       void (*remove)(struct ib_device *);
>
>       struct list_head list;
>    };
>
> by extending the current enum ib_node_type to something like
>
>    enum rdma_device_type {
>       RDMA_DEVICE_IB_CA,
>       RDMA_DEVICE_IB_SWITCH,
>       RDMA_DEVICE_IB_ROUTER,
>       RDMA_DEVICE_RNIC
>    };
>
> Then the various pieces of code layered on top of the RDMA midlayer
> can decide whether they want to deal with a particular device or not
> by looking at the node_type member.  For example, the IB CM, IPoIB,
> etc. could ignore devices of type RDMA_DEVICE_RNIC, while SDP or iSER
> would use all devices and the RNIC CM would take only devices of type
> RDMA_DEVICE_RNIC.
>
> Then someone would have to start implementing a low-level driver for a
> specific RNIC, and find which modifications to the existing verbs are
> required.  For example, I believe the QP attribute structure passed
> into the QP modify verb probably has to become a union containing the
> IB attributes and the RNIC attributes.  However, most verbs should
> work fine with at most trivial modifications.
>
> The existing OpenIB SDP code will be a good example to study as we
> determine what abstractions need to be added to make it simple for
> consumers to deal with the differences between IB and RNIC.
>
>  - R.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050525/d9724571/attachment.html>