[ofa-general] FW: possible rkey byteswap

[Sufficool, Stanley]

The OFED group might want to be included in this discussion. ;)

-----Original Message-----
From: ofw-bounces at lists.openfabrics.org
[mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Fab Tillier
Sent: Thursday, February 07, 2008 11:25 AM
To: Tzachi Dar; frank zago
Cc: ofw at lists.openfabrics.org
Subject: RE: possible rkey byteswap (was Re: [ofw] what's up
withtheOFAWindows project?)


As long as the rkey is exchanged on the wire in network order interop is
not an issue.  However, this requires the client to swap the rkey before
putting it on the wire, as well as after getting it from the wire.

So both the Linux and Windows driver models work, they just work
differently which introduces confusion when people port their
applications.  It might be helpful to have a 'porting how-to guide' that
highlights such differences if the Linux stack can't be changed.

-Fab

-----Original Message-----
From: ofw-bounces at lists.openfabrics.org
[mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Tzachi Dar
Sent: Thursday, February 07, 2008 1:22 AM
To: Fab Tillier; frank zago
Cc: ofw at lists.openfabrics.org
Subject: RE: possible rkey byteswap (was Re: [ofw] what's up with
theOFAWindows project?)

I'm just trying to verify what Fab says:

Is interoperation between Linux big- and little-endian broken? If so,
who can push the fix to the Linux code?

Once that will be done we can start looking at the windows to Linux
interoperation.

Thanks
Tzachi

> -----Original Message-----
> From: ofw-bounces at lists.openfabrics.org 
> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Fab Tillier
> Sent: Thursday, February 07, 2008 1:20 AM
> To: frank zago
> Cc: ofw at lists.openfabrics.org
> Subject: RE: possible rkey byteswap (was Re: [ofw] what's up with 
> theOFAWindows project?)
>
> Ok, let me rephrase - the RKey should be opaque, and should always be 
> in network order (i.e. never ever swapped in any code except the 
> lowest level HW driver if necessary).
>
> In your example, your application would also be broken between the 
> big- and little-endian Linux machines, independent of Windows.  For 
> the big- and little-endian machines to interoperate, they must agree 
> on a wire format for exchanging the RKey.  The application in this 
> case would swap the RKey into network order (and the swap would be a
> noop on the big-endian machine) when it puts it on the wire,
> and swap to host order (again a noop for the big-endian
> machine) when it receives it form the wire.  The Windows
> version of the application would never swap the RKey as it is
> always treated as an opaque network order value.
>
> -Fab
>
> -----Original Message-----
> From: frank zago [mailto:fzago at systemfabricworks.com]
> Sent: Wednesday, February 06, 2008 2:22 PM
> To: Fab Tillier
> Cc: Robert Pearson; 'Tom Tucker'; ofw at lists.openfabrics.org
> Subject: Re: possible rkey byteswap (was Re: [ofw] what's up with the 
> OFAWindows project?)
>
> Fab Tillier wrote:
> > The RKey is always an opaque value - it only has meaning to
> the HCA hardware, and is used in combination with the PD of the QP on 
> which the RETH is received to do the address translation.  It's a 
> token, and should be treated as such. It cannot be interpreted by 
> anything other than the HW/driver that generated it.  The value really

> should never be manipulated outside of the HCA hardware domain (which
> includes the HW and the HW-specific driver).
> >
> > -Fab
> The rkey cannot be opaque. You register a memory region on windows, 
> get a rkey, send it to a big endian linux and a little endian linux 
> host. Both try to use it, and one of them will fail.
>
> I think the application must know what format is this rkey, so they 
> can pass it along while keeping its byte order property. If all IB 
> stacks return the rkey in network order, and accept  rkey also in 
> network order, then there is no interop problem anymore.
>
> Frank.
>
> _______________________________________________
> ofw mailing list
> ofw at lists.openfabrics.org 
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>
_______________________________________________
ofw mailing list
ofw at lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
_______________________________________________
ofw mailing list
ofw at lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw