[ofw] RE: convert_av_handle

[Leonid Keller]

IBAL was designed to work without vendor library, but I never heard
someone tried it.
I believe there are a lot of bugs in this way. 
But why do you need it WITHOUT ?
IBAL is HW-agnostic.
Who will work with *real* HCA if not vendor dll ?
IBAL can't post packets to an unknown HW itself !

> -----Original Message-----
> From: ofw-bounces at lists.openfabrics.org 
> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Usha 
> Srinivasan
> Sent: Thursday, October 22, 2009 8:35 PM
> To: Sean Hefty; ofw at lists.openfabrics.org
> Subject: [ofw] RE: convert_av_handle
> 
> Sean,
> Thanks so much for your clarification.  I understand what's 
> going on now.
> 
> However, I have a question.  Has the current ibal code every 
> been run WITHOUT a vendor's user-mode dll?  Is it possible 
> that when the vendor's dll doesn't exist and h_qp->h_ci_qp is 
> NULL, there is a double av conversion?  Once, done by 
> convert_av_handle in ud_post_send and second time by this 
> code in ual_post_send:
> 
> if( h_qp->type == IB_QPT_UNRELIABLE_DGRM ) {
> 	p_qp_ioctl->in.send_wr[num_wr].dgrm.ud.h_av =
> 
>             (ib_av_handle_t) (ULONG_PTR) p_wr->dgrm.ud.h_av->obj.hdl;
> 
> }
> 
> Because what you have in this UD case is 
> dgrm.ud.h_av->obj.hdl->obj.hdl which cause an access 
> violation.  I wonder if the above if statement should only be 
> run in the case where the vendor's library was loaded.
> 
> Thanks in advance.
> Usha
> 
> -----Original Message-----
> From: Sean Hefty [mailto:sean.hefty at intel.com]
> Sent: Wednesday, October 21, 2009 5:13 PM
> To: Usha Srinivasan; ofw at lists.openfabrics.org
> Subject: RE: convert_av_handle
> 
> list copied on reply:
> 
> >I have a question for you.  I am dying with access violation in 
> >ibald.dll due to this dereference of dgrm.ud.h_av.
> >
> >
> >
> >if( h_qp->type == IB_QPT_UNRELIABLE_DGRM )
> >
> >{
> >
> >p_qp_ioctl->in.send_wr[num_wr].dgrm.ud.h_av =
> >
> >            (ib_av_handle_t) (ULONG_PTR) p_wr->dgrm.ud.h_av->obj.hdl;
> >
> >}
> >
> >
> >
> >It turns out that dgrm.ud.h_av was "converted " in this code:
> >
> >      /* Convert all AV handles for verb provider usage. */
> >
> >      for( p_wr = p_send_wr; p_wr; p_wr = p_wr->p_next )
> >
> >      {
> >
> >            CL_ASSERT( p_wr->dgrm.ud.h_av );
> >
> >            p_wr->dgrm.ud.rsvd = p_wr->dgrm.ud.h_av;
> >
> >            p_wr->dgrm.ud.h_av = convert_av_handle( h_qp, 
> > p_wr->dgrm.ud.h_av );
> >
> >      }
> >
> >
> >
> >      status = h_qp->pfn_ud_post_send(
> >
> >            h_qp->h_ud_send_qp, p_send_wr, pp_send_failure );
> >
> >
> >
> >And, the conversion does this:
> >
> >#define convert_av_handle(h_qp, h_av) \
> >
> >                ((h_qp)->h_ci_qp?(h_av)-
> >>h_ci_av:(ib_av_handle_t)(ULONG_PTR)(h_av)->obj.hdl)
> >
> >
> >
> >I suppose that means that obj.hdl in h_av is not initialized 
> correctly.
> >
> >I don't understand what this av conversion is and I'm trying 
> to figure 
> >out what this means in terms of missing driver 
> functionality.  So, any 
> >help most appreciated.
> 
> IBAL and verbs providers maintain separate structures.  When 
> crossing from the IBAL interface to the verbs provider 
> interface, all handles must be converted.
> This is easily seen when for objects like QPs or CQs.  E.g. 
> IBAL takes a pointer to its QP structure, then calls the 
> verbs provider with its handle.
> ib_query_qp(h_qp ...) becomes ci_query_qp(h_qp->h_ci_qp ...)
> 
> The use of address vectors is similar.  Only in this case, 
> since the IBAL address vector handle is inside the work 
> request, rather than creating an entirely new work request 
> list with the correct verbs provider handle, IBAL simply 
> munges the handle in the original list before giving the WR 
> list to the verbs provider.
> 
> This is the reason that winverbs uses an address vector 'key' 
> in work requests, rather than a handle that must be replaced.
> 
> - Sean
> 
> _______________________________________________
> ofw mailing list
> ofw at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>