[Ofa_boardplus] XWG meeting minutes 11/29

[Woodruff, Robert J]

Ah, I get it. Yes for kernel based things like NFS/RDMA and SRP, experimentation over standard Ethernet would need something like SoftRoCe or Soft iWarp. 
For user-space applications, verbs based experimentation could also use SoftRoce or SoftiWarp or OFI which also has a sockets provider.

Woody


-----Original Message-----
From: Paul Grun [mailto:grun at cray.com] 
Sent: Thursday, November 29, 2018 6:11 PM
To: Jason Gunthorpe <jgg at ziepe.ca>; Doug Ledford <dledford at redhat.com>
Cc: Hefty, Sean <sean.hefty at intel.com>; Woodruff, Robert J <robert.j.woodruff at intel.com>; Jim Ryan <jimdryan at gmail.com>; ofa_boardplus at lists.openfabrics.org
Subject: RE: [Ofa_boardplus] XWG meeting minutes 11/29

This was also my understanding of Doug's original proposal.  The idea is to allow experimentation with 'emulated RDMA hardware', i.e. SoftRoCE, using the verbs API, over an Ethernet network.

>-----Original Message-----
>From: Jason Gunthorpe [mailto:jgg at ziepe.ca]
>Sent: Thursday, November 29, 2018 3:44 PM
>To: Doug Ledford <dledford at redhat.com>
>Cc: Hefty, Sean <sean.hefty at intel.com>; Paul Grun <grun at cray.com>; 
>Woodruff, Robert J <robert.j.woodruff at intel.com>; Jim Ryan 
><jimdryan at gmail.com>; ofa_boardplus at lists.openfabrics.org
>Subject: Re: [Ofa_boardplus] XWG meeting minutes 11/29
>
>At LPC Chunk and Bart were specifically interested in testing verbs 
>based applications like NFS-RDMA and SRP. Not really a space that 
>overlaps with libfabric..
>
>Jason
>
>On Thu, Nov 29, 2018 at 05:34:09PM -0500, Doug Ledford wrote:
>> You can use libfabric over TCP sockets, but it doesn't have the feel 
>> of RDMA hardware.  What I was going for was something that a user 
>> could do to emulate having the hardware.  This provides a platform 
>> for exploring all of the aspects related to RDMA, including things 
>> like configuration of devices, seeing how the OS reacts to the 
>> hardware, as well as programming *any* of the RDMA APIs.  This is why 
>> I was most interested in soft-iWARP or soft-RoCE.
>>
>> On 11/29/2018 5:20 PM, Hefty, Sean wrote:
>> > Woody wasn't referring to the verbs provider.  He was talking about 
>> > using
>sockets over tcp.
>> >
>> > RDMA != VERBS   !!!!!
>> >
>> > If OFA still can't get this right, then, yes, there's no way anyone 
>> > outside of the
>field is ever going to understand it.
>> >
>> > OFA has already made the investment.  It's harmless, been available 
>> > for years,
>and is free.
>> >
>> >
>> >> You’re quite right Woody; I hadn’t thought of using libfabric with 
>> >> the verbs provider running over Ethernet.
>> >>
>> >>
>> >>
>> >> Another approach is the ‘cluster on a laptop’ that I described.
>> >> That one uses KVM and standard enet virtual switches to create a 
>> >> ‘cluster in a box’.
>> >>
>> >>
>> >>
>> >> In either case, Doug’s point was that the OFA is in a position to 
>> >> make it easier for people to get exposed to RDMA in a harmless and 
>> >> zero cost way.  I think that’s something in which the OFA should invest.
>> >> Fastest way I can imagine to accelerate the adoption of high 
>> >> performance networks.
>> >>
>> >> -Paul
>> >>
>> >>
>> >>
>> >> From: Woodruff, Robert J [mailto:robert.j.woodruff at intel.com]
>> >> Sent: Thursday, November 29, 2018 1:59 PM
>> >> To: Paul Grun <grun at cray.com>; Jim Ryan <jimdryan at gmail.com>
>> >> Cc: ofa_boardplus at lists.openfabrics.org,
>> >> <ofa_boardplus at lists.openfabrics.org>
>> >> Subject: RE: [Ofa_boardplus] XWG meeting minutes 11/29
>> >>
>> >>
>> >>
>> >>> There’s no easy way to experiment with RDMA and learn about it,
>> >> without buying h/w.
>> >>
>> >>
>> >>
>> >> This is not exactly true. With OFI, it supports RDMA and we have a 
>> >> sockets provider that people can use with any off the shelf 
>> >> Ethernet NIC to experiment and learn about RDMA.
>> >>
>> >> In fact, we have a lot of development internally that uses the OFI 
>> >> sockets provider for initial development and then later test and 
>> >> tune it on hardware accelerated RDMA NICs.
>> >>
>> >>
>> >>
>> >> Woody
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> From: Ofa_boardplus [mailto:ofa_boardplus- 
>> >> bounces at lists.openfabrics.org] On Behalf Of Paul Grun
>> >> Sent: Thursday, November 29, 2018 1:24 PM
>> >> To: Jim Ryan <jimdryan at gmail.com <mailto:jimdryan at gmail.com> >
>> >> Cc: ofa_boardplus at lists.openfabrics.org
>> >> <mailto:ofa_boardplus at lists.openfabrics.org> , 
>> >> <ofa_boardplus at lists.openfabrics.org
>> >> <mailto:ofa_boardplus at lists.openfabrics.org> >
>> >> Subject: Re: [Ofa_boardplus] XWG meeting minutes 11/29
>> >>
>> >>
>> >>
>> >> A tiny nit – the second to approve the minutes was Christoph.
>> >>
>> >>
>> >>
>> >> Another small-ish nit - People are dismayed at how hard it is to 
>> >> test things RDMA-related especially by newbies, complicated by the 
>> >> need to buy HW.
>> >>
>> >> I think it’s a little more than just testing.  There’s no easy way 
>> >> to experiment with RDMA and learn about it, without buying h/w.
>> >>
>> >>
>> >>
>> >> Not-so-small nit - Gilad explained openRDMA doesn’t compete with 
>> >> the OFA. Rather it provides help for those wanting to use RDMA.
>> >> They are not doing new development.
>> >>
>> >> Rather, Gilad asserted that OpenRDMA doesn’t compete with the OFA 
>> >> because it is not doing new development. This is incorrect because 
>> >> the OFA’s mission is not limited to development.  Our mission is 
>> >> to accelerate the development and adoption of (e.g. RDMA).  No 
>> >> good argument was offered for why OpenRDMA should not be hosted by 
>> >> the OFA under the auspices of the existing OFVWG (which was 
>> >> originally requested by MLNX and is currently chaired by MLNX).
>> >>
>> >>
>> >>
>> >> The OFA was originally formed specifically to foster the 
>> >> development, support, and adoption of RDMA and stands ready to 
>> >> continue
>doing so.
>> >> Nothing about that has changed in our mission.
>> >>
>> >>
>> >>
>> >> -Paul
>> >>
>> >>
>> >>
>> >> From: Ofa_boardplus [mailto:ofa_boardplus- 
>> >> bounces at lists.openfabrics.org] On Behalf Of Jim Ryan
>> >> Sent: Thursday, November 29, 2018 12:30 PM
>> >> To: ofa_boardplus at lists.openfabrics.org
>> >> <mailto:ofa_boardplus at lists.openfabrics.org> , 
>> >> <ofa_boardplus at lists.openfabrics.org
>> >> <mailto:ofa_boardplus at lists.openfabrics.org> >
>> >> Subject: [Ofa_boardplus] XWG meeting minutes 11/29
>> >>
>> >>
>> >>
>> >> A very complex meeting, I hope I got it right
>> >>
>> >>
>> >>
>> >> Jim
>> >
>> > _______________________________________________
>> > Ofa_boardplus mailing list
>> > Ofa_boardplus at lists.openfabrics.org
>> > https://lists.openfabrics.org/mailman/listinfo/ofa_boardplus
>> >
>>
>>
>
>
>
>
>> _______________________________________________
>> Ofa_boardplus mailing list
>> Ofa_boardplus at lists.openfabrics.org
>> https://lists.openfabrics.org/mailman/listinfo/ofa_boardplus