[openib-general] mthca and DDR not hidden

[Hal Rosenstock]

On Thu, 2004-09-30 at 15:10, Roland Dreier wrote:
>     Hal> I attempted to send UD and got stopped by the following in mthca_av.c:
> 
>     Hal> int mthca_read_ah(struct mthca_dev *dev, struct mthca_ah *ah,
>     Hal>                   struct ib_ud_header *header)
>     Hal> {
>     Hal>         if (ah->on_hca)
>     Hal>                 return -EINVAL;
> 
>     Hal> on_hca is set due to the following in mthca_create_ah:
> 
>     Hal>        if (!atomic_read(&pd->sqp_count) &&
>     Hal>             !(dev->mthca_flags & MTHCA_FLAG_DDR_HIDDEN)) {
> 
>     [...]
> 
>     Hal>                 ah->on_hca = 1;
> 
>     Hal> Is it a requirement to run the HCA with DDR hidden ? 
> 
> Notice that on_hca won't be set if either the DDR is hidden _or_ the
> PD being used has special QPs attached.  We only need to reread the AH
> for sends on QP0/QP1 (not general UD QPs), so this shouldn't be a
> problem in practice.  Are you using the same PD for your QPs and AHs
> (they have to match for correct operation)?

Oops. Looks like we need a way to expose the PD and MR to be able to do
this. How about adding this into the mad_agent structure returned ? If
that makes sense, I will generate the patch for this.

> I run nearly all of my HCAs without the DDR hidden and it works fine.
> 
>     Hal> Is there a way to get the AH when the DDR is not hidden ? If so, should
>     Hal> this support be added into mthca ?
> 
> Yes, it's possible.  However I don't think it's urgent because things
> should work fine as it stands now.

Agreed (and there already is a TODO list item on DDR hidden relative to
UD AHs (other than QP0/1)).

> The only issue seems to be that we
> may not generate the correct error when an AH from a different PD is
> used on QP0/QP1.

What's should be the correct error for this case ?

Thanks.

-- Hal