[openib-general] [PATCH 0/5] iw_cxgb3 - misc cleanup and fixes

[Steve Wise]

On Fri, 2007-02-09 at 08:23 -0600, Steve Wise wrote:
> On Fri, 2007-02-09 at 08:51 +0200, Michael S. Tsirkin wrote:
> > > > Also I agree with MST, I would like to see the core/ subdirectory die
> > > > completely.
> > > > 
> > > 
> > > ok ok...I'll kill the subdir...
> > 
> > It's not just the directory BTW. Stuff like building completions in
> > t3_cqe format and then reformatting to ib_wc seems to be much more confusing
> > (and some of it is actually on datapath).
> 
> The t3_cqe format is built BY THE HW.
> 
> 
> > Same goes for t3_wq and I suspect everything else defined in cxio_wr.h -
> > please, use the native types from include/rdma/.
> > 
> 
> Ditto.  t3_wq is the HW format.
> 

To be more precise:  

struct t3_wq is the struct used to describe the T3 HW WQ, which is both
the SQ and RQ for the QP.  

struct t3_cq is the struct used to describe the T3 HW CQ -and- a SW CQ
used to maintain proper completion ordering that isn't maintained by the
HW for some operations.  

union t3_wr defines the union of all the HW-specific WR structs.

struct t3_cqe defines the HW CQE format.

All of the is very tightly integrated with the HW.  

These HW-specific structs  are included in a high-level struct that
defines the object and has all the stuff needed to integrate into the
OFA stack.

Example:

struct iwch_qp defines the top-level QP structure that maintains both
the T3 HW struct (struct t3_wq) and the OFA struct (struct ib_qp) as
well as attributes, wait objects, locks, etc to correctly implement the
OFA QP object.   

This is similar to what other providers do:

hw/mthca/mthca_provider.h: mthca_qp includes 2 mthca_wq structs for the
SQ and RQ.

hw/amso/c2_provider.h: c2_qp includes 2 c2_mq structs for the SQ and RQ
message queues.

hw/ipath/ipath_verbs.h: ipath_qp include ipath_swqe and ipath_rq for
their work queues.

WRT data path operations, consider iwch_poll_cq_one().  The CQE is in T3
FW format and must be converted into a OFA struct ib_wc.  There's no way
around this, right?  mthca_poll_one() does the same thing.  Ditto for
c2_poll_one().


Hope this helps.  

Steve.