[ewg] [PATCH] libibverbs: add raw ethernet QP type IBV_QPT_RAW_ETH=7

[Walukiewicz, Miroslaw]

Adding a IBV_QPT_RAW_ETH enables a L2 traffic kernel bypass  using user-space verbs API. 

The L2 RAW_ETH acceleration assumes that user application transmits and receives a whole L2 frame including MAC/IP/UDP/TCP headers. Depending on frame content and available HW filters in HW any L2 traffic type could be accelerated.

The sample code for IBV_QPT_RAW_ETH is provided by NES as implementation of  IPv4 multicast acceleration. 

In this sample, first the application creates IBV_QPT_RAW_ETH QP with associated CQ,PD, completion channels as it is performed for RDMA connection.

Next step is enabling L2 MAC address RX filters for directing received multicasts to the RAW_ETH QPs using ibv_attach_multicast() verb. 

>From this point the application is ready to receive and transmit the L2 traffic. 

In multicast acceleration the user application passes to ibv_post_send() the whole IGMP frame including MAC header, IP header, UDP header and UDP payload. It is a user responsibility to make IP fragmentaion when required payload to send is bigger than MTU. Every fragment is a separate L2 frame to transmit. The ibv_poll_cq() provides an information about the status of transmit buffer. 

On receive path when ibv_poll_cq() provides an information about received L2 packet, the Rx buffer (previously posted by ibv_post_recv() ) contains a whole L2 frame including MAC header, IP header and UDP header. It is a user application responsibility to check if received packet is a valid UDP frame so the fragments must be checked and checksums must be computed. 

For me the additional patches in kernel are not necessary because  the functional meaning of the IB_QPT_RAW_ETH for IWARP/Ethernet cards and IB_QPT_RAW_ETY for IB (that is not used as I know) are very similar and there is no need to multiply QP types that have the same meaning. Similarly there is no separate definitions for RC_QP for iWARP and IB, because their meaning is similar. 

Regards,

Mirek Walukiewicz

-----Original Message-----
From: Aleksey Senin [mailto:alekseys at voltaire.com] 
Sent: Sunday, May 02, 2010 3:15 PM
To: Walukiewicz, Miroslaw
Cc: rdreier at cisco.com; ewg at openfabrics.org; Moni Shoua; Yiftah Shahar
Subject: Re: [ewg] [PATCH] libibverbs: add raw ethernet QP type IBV_QPT_RAW_ETH=7

Could you add additional details about its usage in the feature to
community? Why and how can it be used? I understand, that using this QP
type, user will have full control on the packet content and it may be
required when you are missing some features now, like UD, and want
implement them by yourself.
Another thing. Is seems that you have to patch kernel too in order
implement such support. Something like this.

enum ib_qp_type {
        /*
         * IB_QPT_SMI and IB_QPT_GSI have to be the first two entries
         * here (and in that order) since the MAD layer uses them as
         * indices into a 2-entry table.
         */
        IB_QPT_SMI,
        IB_QPT_GSI,

        IB_QPT_RC,
        IB_QPT_UC,
        IB_QPT_UD,
        IB_QPT_XRC,
        IB_QPT_RAW_IPV6,
        IB_QPT_RAW_ETH,
        IB_QPT_RAW_ETY
};
And its, probably, a reason, why IBV_QPT_RAW_ETH should be equal to 7.

On Fri, 2010-04-30 at 18:46 +0200, Mirek Walukiewicz wrote:
> The patch enables usage of L2 raw ethernet QP type for user-space applications.
> 
> Miroslaw Walukiewicz
> 
> 
> Signed-off-by: Mirek Walukiewicz <miroslaw.walukiewicz at intel.com>
> ---
>  include/infiniband/verbs.h |    3 ++-
>  1 files changed, 2 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h
> index 0f1cb2e..55cea89 100644
> --- a/include/infiniband/verbs.h
> +++ b/include/infiniband/verbs.h
> @@ -390,7 +390,8 @@ struct ibv_srq_init_attr {
>  enum ibv_qp_type {
>  	IBV_QPT_RC = 2,
>  	IBV_QPT_UC,
> -	IBV_QPT_UD
> +	IBV_QPT_UD,
> +	IBV_QPT_RAW_ETH = 7
>  };
>  
>  struct ibv_qp_cap {
> 
>  
> _______________________________________________
> ewg mailing list
> ewg at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg