[openib-general] mthca and DDR not hidden

[Roland Dreier]

    Hal> I attempted to send UD and got stopped by the following in mthca_av.c:

    Hal> int mthca_read_ah(struct mthca_dev *dev, struct mthca_ah *ah,
    Hal>                   struct ib_ud_header *header)
    Hal> {
    Hal>         if (ah->on_hca)
    Hal>                 return -EINVAL;

    Hal> on_hca is set due to the following in mthca_create_ah:

    Hal>        if (!atomic_read(&pd->sqp_count) &&
    Hal>             !(dev->mthca_flags & MTHCA_FLAG_DDR_HIDDEN)) {

    [...]

    Hal>                 ah->on_hca = 1;

    Hal> Is it a requirement to run the HCA with DDR hidden ? 

Notice that on_hca won't be set if either the DDR is hidden _or_ the
PD being used has special QPs attached.  We only need to reread the AH
for sends on QP0/QP1 (not general UD QPs), so this shouldn't be a
problem in practice.  Are you using the same PD for your QPs and AHs
(they have to match for correct operation)?

I run nearly all of my HCAs without the DDR hidden and it works fine.

    Hal> Is there a way to get the AH when the DDR is not hidden ? If so, should
    Hal> this support be added into mthca ?

Yes, it's possible.  However I don't think it's urgent because things
should work fine as it stands now.  The only issue seems to be that we
may not generate the correct error when an AH from a different PD is
used on QP0/QP1.

 - Roland