[openib-general] [RFC/BUG] libibverbs: DMA vs. CQ race

[Roland Dreier]

 > Assuming that something along the lines of the previous patch
 > is used, we need to address userspace/kernel compatibility.
 > 
 > The existing abi versioning doesn't seem to be exactly what
 > we want to use, though, because we want to change a verb's
 > semantics to work around a bug. (Changing the abi_version
 > may be an inevitable result, though.)
 > 
 > How about adding "semantic flags" to the mthca_* commands
 > (mthca_create_cq, etc.)? Userspace could read the contents of
 > a new sysfs file which, if found, would indicate the flags
 > that the kernel understands. Then it could pass the flags, if
 > it chooses, to get the kernel to use the desired semantics.

This is not really the design philosophy that we've used in defining
the user-kernel interfaces for IB verbs.  Rather than having
complexity in the kernel to handle both old and new ways of doing
things, the way we've used to handle cases like this is the following:

 - specify new fixed ABI (in this case, mthca abi_version 2)
 - update library to handle old and new ABI (in this case, update
   libmthca to use mthca kernel abi 1 or 2 depending on what it
   detects at runtime)
 - update kernel to implement new ABI, and remove old ABI from kernel
   (in this case, update kernel mthca driver to abi_version 2)

The net effect of this is that updated userspace works fine with any
kernel, but updating the kernel will require updating userspace
libraries too.  However the important point is that once userspace is
updated, it's still possible to boot into old kernels and have things
work without downgrading userspace.

If we really wanted to export some flags from mthca back to libmthca,
I guess it would be possible to bump the abi version and add a flags
field to the response to the alloc_ucontext command, but in this case
I don't see a reason to worry about it.

 - R.