[rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support

[Olaf Kirch]

On Tuesday 15 July 2008 07:33:05 Richard Frank wrote:
> This all seems to be a bit restrictive - which will require some changes in 
> how Oracle uses the IPC for rdma operations - but we need to start somewhere 
> so this seems reasonable.

The best we can do right now for iWARP is to require the app layer to
use at least part of the cmsg interface. You can continue to use GET_MR
and FREE_MR, but you have to tell the kernel about the r_key it's sending
in a message by using CMSG_DEST.

> If we can get bcopy running with Oracle - that'd be a great goal for 1.4. If

I have the patches queued up, and will submit them to Vlad today.

> it's going to require changes in Oracle to use IWARP RDMA - then we're 
> talking about a later Orace release to use it...
> 
> I'm out on vacation will be back in two weeks..

Enjoy!

Olaf

> 
> Rick
> 
> ----- Original Message ----- 
> From: "Olaf Kirch" <olaf.kirch at oracle.com>
> To: "Or Gerlitz" <ogerlitz at voltaire.com>
> Cc: <rds-devel at oss.oracle.com>; <general at lists.openfabrics.org>
> Sent: Tuesday, July 08, 2008 8:00 AM
> Subject: Re: [rds-devel] [ofa-general] [PATCH RFC] rds: add iwarp support
> 
> 
> > On Tuesday 08 July 2008 12:11:40 Or Gerlitz wrote:
> >> Olaf Kirch wrote:
> >> > Okay, there's a third way to fix this, which may even work with today's 
> >> > Oracle IPC code. The hack is to not submit the fastreg WR until we post 
> >> > a SEND that transmits the rkey to a peer. Then
> >> >
> >> >  - we have a QP to post the WRs to
> >> >  - we post the fastreg and the SEND to the same QP,
> >> > ensuring that the peer doesn't initiate the RDMA
> >> > before we've set up the memory registration.
> >> Yes, as Steve noted, using the same QP for the fastreg and send that
> >> carries the rkey/stag is a must since the send should be fenced on the
> >> fastreg completion to make sure the remote side uses a valid key.
> >
> > One theoretical problem is what happens if the application wants to
> > register a memory range and send the rkey to two peers at the same time?
> > We would have to stall the second send explicitly as long as we
> > haven't seen the completion for the first send.
> >
> > Olaf
> > -- 
> > Olaf Kirch  |  --- o --- Nous sommes du soleil we love when we play
> > okir at lst.de |    / | \   sol.dhoop.naytheet.ah kin.ir.samse.qurax
> > _______________________________________________
> > general mailing list
> > general at lists.openfabrics.org
> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >
> > To unsubscribe, please visit 
> > http://openib.org/mailman/listinfo/openib-general
> > 
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
> 



-- 
Olaf Kirch  |  --- o --- Nous sommes du soleil we love when we play
okir at lst.de |    / | \   sol.dhoop.naytheet.ah kin.ir.samse.qurax