[openib-general] [BUG] Byte swapped Pkeys

[Roland Dreier]

    Sean> I think it would make the interface much easier to deal
    Sean> with.  Today, clients must deal with these values in two
    Sean> different formats, depending on how they get their data.
    Sean> It's also not clear what format clients should use.
    Sean> Glancing at the stack, it seems that clients receive data in
    Sean> big endian format via MADs, convert it to host order, then
    Sean> have it converted back into big endian format for use with
    Sean> the hardware.

Mostly true, although for mthca things like QPN are used (in host
order) as indexes into HCA context tables, so passing values like QPN
in big endian format is going to add a fair amount of swapping to mthca.

Basically this all goes back to Mellanox's original choice to use host
byte order.  I guess we don't care about VAPI compatibility any more
but I think a lot of people are used to the byte ordering.

    Sean> Comparisons in little/big endian format doesn't matter, as
    Sean> long as the data is consistent.

Mostly true although eg CM peer compare requires QPN in host order. 

    Sean> I don't really have time for this now either, but I do think
    Sean> removing unnecessary byte-swapping is the right thing to do.

If this is something we want to do, I think the sanest way to go about
it is to use the new __be16, __be32 etc. types (in current Linus bk,
will be in kernel 2.6.9) and make sure the whole tree is sparse
clean.  For example change all P_Key APIs to take __be16 and then go
through and fix all the sparse warnings.

 - R.