[ofw] Re: ib_types.h moving

[Ira Weiny]

On Thu, 1 Oct 2009 14:47:33 -0700
"Sean Hefty" <sean.hefty at intel.com> wrote:

> >So far I can only think of 2 "correct" options.
> >
> >   1) Make ib_types.h into a new library and have libibverbs, libibmad,
> >      opensm, and MPI's use the constants and helper functions there.  This of
> >      course would force some dependencies.
> >
> >   2) split up and spread ib_types.h around to appropriate places.  This will
> >      likely include libibmad and libibverbs (others?) I don't think libibumad
> >      is appropriate really.  At first, I suggested this mainly out of
> >      convenience because WinOF uses it.  However, I don't think it is
> >      architecturally correct.  libibumad looks to have a sound interface and
> >      should not be messed with.
> 
> The only issue that was pointed out was that the IB ACM duplicated some network
> structure definitions (the SA MAD format, McMemberRecord, PathRecord).  It seems
> perfectly natural to me to have the library that allows you to send and receive
> MADs to provide these definitions, especially considering how fundamental these
> specific attributes are to using IB.  Even though umad_send and umad_recv are
> both defined to take a void * input buffer, they expect a specific format to
> that buffer.

Combine libibmad into libibumad?  Or just the core structure definitions?  I
will admit that accepting void * seems wrong when you know it is
supposed to be a mad packet.  In fact both umad_send and umad_recv imediately
cast the void * to struct ib_user_mad *!  Why not specify that in the
parameter type?

> 
> libibverbs defines host structure definitions for McMemberRecord and PathRecord,
> but doesn't use them.  This allows the rdma_cm and ib_cm to remain independent,
> but use the same definition.

:-(  I missed those.  Is there a reason they must remain independent?  (I
assume you mean independent of the mad libs?)

> 
> IB ACM is coded to libibumad directly, mainly because I didn't see any benefit
> to coding to a higher level MAD interface that was more complex to use.  Trying
> to keep the defines in a higher level library, which is what we have now, either
> results in duplicating them or forcing a dependency where there wouldn't be any.

I understand the desire to avoid using libibmad.  Perhaps you are correct that
there are some things which are "core" MAD functionality, structure
definitions for example, which should be at a lower level.  However, I think
it is still appropriate to put some things from ib_types into libibverbs and
to expect people to use those definitions when required.

> 
> And to be clear, I'm only referring to definitions in ib_types.h (and some in
> mad.h), and not any actual functionality.
> 

<sigh> yes this discussion has digressed.  But I don't think we should dive
into breaking up (or moving) ib_types just to move things around.  I think
Sasha is right we need to have a plan for where things _should_ end up.  As I
look for homes for all the items of ib_types I find that it is not just one
place.  :-/

Ira

-- 
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
weiny2 at llnl.gov