[openib-general] makiing ibverb.h transport neutral -- 2nd draft

[Caitlin Bestler]

> -----Original Message-----
> From: Sean Hefty [mailto:mshefty at ichips.intel.com] 
> Sent: Wednesday, July 13, 2005 12:05 PM
> To: Caitlin Bestler
> Cc: openib-general; rdma-developers at lists.sourceforge.net
> Subject: Re: [openib-general] makiing ibverb.h transport 
> neutral -- 2nd draft

> 
> > The alternative would be to use extensivel sub-types on input, and 
> > unions for work completions. That works, but could easily 
> be confusing 
> > to developers.
> 
> I'm not sure that's any more confusing than requiring the 
> developer to know which data type is actually expected.  It 
> does make the data structures larger than they may need to be.
> 

The consumer will know which transport. But will it want
to do an "if (ib) ... else (iwarp) ..." wrapper just to
minimize the size of the work completion buf? I doubt it.
The work completions delivered to the user are unlikely
to be kept in arrays. More likely they will be immediately
translated to a middleware event (DAPL, MPI, RPC ...).

And even if the transport is known, the caller will frequently
not know the type (receive, send, bind ...) anyway.

Now it would be real nifty if C defined an "extensible union"
where various different include files could add variants to 
a parent type, and then application code could use the parent
type knowing that it would be large enough for the worst case
and yet only have to know about the variants that the specific
application actually wanted to deal with.

But since ANSI C does not have such a construct we can either
force *all* the variants to be visible in a big overbloated
union (but at least know that they are all there), or we can
allow sub-types to be declared as necessary and hope that all
sub-type declarers double-check that their sizeof() fits within
an allotted maximum.

The current draft follows the philosophy of avoiding overbloated
centralized unions, which seemed to be the consensus from the
first round of discussion.

The overbloated union also works, but it has its own pains.
I have no problem changing it back to the union approach on 
either the forward and/or reverse path, if that is the consensus.
Personally, I think using a different style for forware and
reverse would be more confusing, that using a cast in response
to an opcode is no more inherently suspect than selecting a
union sub-variant in response to an opcode and that many or
most applications will need to support multiple work completion
sub-types anyway.