[ofw] RE: convert_av_handle

[Usha Srinivasan]

Sean,  
Thanks so much for your clarification.  I understand what's going on now.

However, I have a question.  Has the current ibal code every been run WITHOUT a vendor's user-mode dll?  Is it possible that when the vendor's dll doesn't exist and h_qp->h_ci_qp is NULL, there is a double av conversion?  Once, done by convert_av_handle in ud_post_send and second time by this code in ual_post_send:

if( h_qp->type == IB_QPT_UNRELIABLE_DGRM )
{
	p_qp_ioctl->in.send_wr[num_wr].dgrm.ud.h_av =

            (ib_av_handle_t) (ULONG_PTR) p_wr->dgrm.ud.h_av->obj.hdl;

}

Because what you have in this UD case is dgrm.ud.h_av->obj.hdl->obj.hdl which cause an access violation.  I wonder if the above if statement should only be run in the case where the vendor's library was loaded.

Thanks in advance.
Usha

-----Original Message-----
From: Sean Hefty [mailto:sean.hefty at intel.com] 
Sent: Wednesday, October 21, 2009 5:13 PM
To: Usha Srinivasan; ofw at lists.openfabrics.org
Subject: RE: convert_av_handle

list copied on reply:

>I have a question for you.  I am dying with access violation in ibald.dll due
>to this dereference of dgrm.ud.h_av.
>
>
>
>if( h_qp->type == IB_QPT_UNRELIABLE_DGRM )
>
>{
>
>p_qp_ioctl->in.send_wr[num_wr].dgrm.ud.h_av =
>
>            (ib_av_handle_t) (ULONG_PTR) p_wr->dgrm.ud.h_av->obj.hdl;
>
>}
>
>
>
>It turns out that dgrm.ud.h_av was "converted " in this code:
>
>      /* Convert all AV handles for verb provider usage. */
>
>      for( p_wr = p_send_wr; p_wr; p_wr = p_wr->p_next )
>
>      {
>
>            CL_ASSERT( p_wr->dgrm.ud.h_av );
>
>            p_wr->dgrm.ud.rsvd = p_wr->dgrm.ud.h_av;
>
>            p_wr->dgrm.ud.h_av = convert_av_handle( h_qp, p_wr->dgrm.ud.h_av );
>
>      }
>
>
>
>      status = h_qp->pfn_ud_post_send(
>
>            h_qp->h_ud_send_qp, p_send_wr, pp_send_failure );
>
>
>
>And, the conversion does this:
>
>#define convert_av_handle(h_qp, h_av) \
>
>                ((h_qp)->h_ci_qp?(h_av)-
>>h_ci_av:(ib_av_handle_t)(ULONG_PTR)(h_av)->obj.hdl)
>
>
>
>I suppose that means that obj.hdl in h_av is not initialized correctly.
>
>I don't understand what this av conversion is and I'm trying to figure out what
>this means in terms of missing driver functionality.  So, any help most
>appreciated.

IBAL and verbs providers maintain separate structures.  When crossing from the
IBAL interface to the verbs provider interface, all handles must be converted.
This is easily seen when for objects like QPs or CQs.  E.g. IBAL takes a pointer
to its QP structure, then calls the verbs provider with its handle.
ib_query_qp(h_qp ...) becomes ci_query_qp(h_qp->h_ci_qp ...)

The use of address vectors is similar.  Only in this case, since the IBAL
address vector handle is inside the work request, rather than creating an
entirely new work request list with the correct verbs provider handle, IBAL
simply munges the handle in the original list before giving the WR list to the
verbs provider.

This is the reason that winverbs uses an address vector 'key' in work requests,
rather than a handle that must be replaced.

- Sean