possible rkey byteswap (was Re: [ofw] what's up with theOFAWindows project?)

[Fab Tillier]

As long as the rkey is exchanged on the wire in network order interop is not an issue.  However, this requires the client to swap the rkey before putting it on the wire, as well as after getting it from the wire.

So both the Linux and Windows driver models work, they just work differently which introduces confusion when people port their applications.  It might be helpful to have a 'porting how-to guide' that highlights such differences if the Linux stack can't be changed.

-Fab

-----Original Message-----
From: ofw-bounces at lists.openfabrics.org [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Tzachi Dar
Sent: Thursday, February 07, 2008 1:22 AM
To: Fab Tillier; frank zago
Cc: ofw at lists.openfabrics.org
Subject: RE: possible rkey byteswap (was Re: [ofw] what's up with theOFAWindows project?)

I'm just trying to verify what Fab says:

Is interoperation between Linux big- and little-endian broken?
If so, who can push the fix to the Linux code?

Once that will be done we can start looking at the windows to Linux
interoperation.

Thanks
Tzachi

> -----Original Message-----
> From: ofw-bounces at lists.openfabrics.org
> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Fab Tillier
> Sent: Thursday, February 07, 2008 1:20 AM
> To: frank zago
> Cc: ofw at lists.openfabrics.org
> Subject: RE: possible rkey byteswap (was Re: [ofw] what's up
> with theOFAWindows project?)
>
> Ok, let me rephrase - the RKey should be opaque, and should
> always be in network order (i.e. never ever swapped in any
> code except the lowest level HW driver if necessary).
>
> In your example, your application would also be broken
> between the big- and little-endian Linux machines,
> independent of Windows.  For the big- and little-endian
> machines to interoperate, they must agree on a wire format
> for exchanging the RKey.  The application in this case would
> swap the RKey into network order (and the swap would be a
> noop on the big-endian machine) when it puts it on the wire,
> and swap to host order (again a noop for the big-endian
> machine) when it receives it form the wire.  The Windows
> version of the application would never swap the RKey as it is
> always treated as an opaque network order value.
>
> -Fab
>
> -----Original Message-----
> From: frank zago [mailto:fzago at systemfabricworks.com]
> Sent: Wednesday, February 06, 2008 2:22 PM
> To: Fab Tillier
> Cc: Robert Pearson; 'Tom Tucker'; ofw at lists.openfabrics.org
> Subject: Re: possible rkey byteswap (was Re: [ofw] what's up
> with the OFAWindows project?)
>
> Fab Tillier wrote:
> > The RKey is always an opaque value - it only has meaning to
> the HCA hardware, and is used in combination with the PD of
> the QP on which the RETH is received to do the address
> translation.  It's a token, and should be treated as such.
> It cannot be interpreted by anything other than the HW/driver
> that generated it.  The value really should never be
> manipulated outside of the HCA hardware domain (which
> includes the HW and the HW-specific driver).
> >
> > -Fab
> The rkey cannot be opaque. You register a memory region on
> windows, get a rkey, send it to a big endian linux and a
> little endian linux host.
> Both try to use it, and one of them will fail.
>
> I think the application must know what format is this rkey,
> so they can pass it along while keeping its byte order
> property. If all IB stacks return the rkey in network order,
> and accept  rkey also in network order, then there is no
> interop problem anymore.
>
> Frank.
>
> _______________________________________________
> ofw mailing list
> ofw at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>
_______________________________________________
ofw mailing list
ofw at lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw