possible rkey byteswap (was Re: [ofw] what's up with the OFAWindows project?)

[Fab Tillier]

Ok, let me rephrase - the RKey should be opaque, and should always be in network order (i.e. never ever swapped in any code except the lowest level HW driver if necessary).

In your example, your application would also be broken between the big- and little-endian Linux machines, independent of Windows.  For the big- and little-endian machines to interoperate, they must agree on a wire format for exchanging the RKey.  The application in this case would swap the RKey into network order (and the swap would be a noop on the big-endian machine) when it puts it on the wire, and swap to host order (again a noop for the big-endian machine) when it receives it form the wire.  The Windows version of the application would never swap the RKey as it is always treated as an opaque network order value.

-Fab

-----Original Message-----
From: frank zago [mailto:fzago at systemfabricworks.com]
Sent: Wednesday, February 06, 2008 2:22 PM
To: Fab Tillier
Cc: Robert Pearson; 'Tom Tucker'; ofw at lists.openfabrics.org
Subject: Re: possible rkey byteswap (was Re: [ofw] what's up with the OFAWindows project?)

Fab Tillier wrote:
> The RKey is always an opaque value - it only has meaning to the HCA hardware, and is used in combination with the PD of the QP on which the RETH is received to do the address translation.  It's a token, and should be treated as such.  It cannot be interpreted by anything other than the HW/driver that generated it.  The value really should never be manipulated outside of the HCA hardware domain (which includes the HW and the HW-specific driver).
>
> -Fab
The rkey cannot be opaque. You register a memory region on windows, get
a rkey, send it to a big endian linux and a little endian linux host.
Both try to use it, and one of them will fail.

I think the application must know what format is this rkey, so they can
pass it along while keeping its byte order property. If all IB stacks
return the rkey in network order, and accept  rkey also in network
order, then there is no interop problem anymore.

Frank.