[openib-general] makiing ibverb.h transport neutral -- 2nd draft

[Sean Hefty]

Caitlin Bestler wrote:
> So would you favor changing the signature to match the largest
> size, with  comment to the effect that you *could* use a cast
> and a smaller object but only if you are sure.

I haven't decided on what might be the best approach for this call yet, I'd 
just like to ensure that the API provides the best performance.  If QPs and 
CQs are both transport specific, I'm wondering now if verbs is the best 
place to abstract RDMA functionality, or if it will result in losing some of 
the details provided by the underlying architecture.

There doesn't seem to be much to iwarp_verbs.h.  Assuming that the API is 
complete, iWarp seems to use a subset of the structures defined by IB.  Why 
couldn't the current data structures just be renamed from ib_blah to 
rdma_blah, with notes that some fields apply only to IB?

Some of the data structures already have fields that only apply for specific 
completion types, work requests, QPs, etc.  So, marking them for IB only 
doesn't seem like that much of a stretch, and transport neutral code would 
need to allocate the space anyway.  This way we would only need to change a 
few enums and move IB specific functions to a new file.

> I can see the danger of someone just using 'rdma_wc', so flipping
> the naming convention would definitely make sense.
> 
> As to the size. Is there a size smaller than 256 that crosses
> over into a significantly safer stack allocation but is still
> *very* safe in terms of 'sure to be big enough'? Keep in mind
> that 'struct sockaddr' has problems precisely because 'biggest
> possible address' turned out to be too small.

Work completions are used in speed path operations, so should be optimized 
for caching, retrieving multiple completions, etc., versus optimizing it for 
transport neutral code.  struct rdma_wc_storage isn't used anywhere in the 
API, so why not make that a union of struct ib_wc and struct iwarp_wc?  (Of 
course this leads us back to unions if swapping the naming convention...)

- Sean