[openib-general] DMA mapping abuses in MAD layer

[Roland Dreier]

    Sean> It sounds like we need to change how the mapping is done.
    Sean> Can we let the MAD layer always control the mapping?

I guess so.  Another alternative would be for the consumer to provide
some sort of callback interface to handle the mapping, but that
doesn't seem feasible.

    Sean> We really just need to change the post_send_mad routine,
    Sean> don't we?

I guess so -- and remove the DMA mapping call from ib_create_send_mad().

    Sean> The original intent around that API was to permit posting
    Sean> the WR directly onto the QP.  Since this isn't the case,
    Sean> what about changing post send to take as input an
    Sean> ib_mad_send_buf, with the work request and SGE fields
    Sean> removed?

We probably still want to handle gather lists for posting sends I
think.  Another (rather unrelated) issue that I just noticed the other
day is that something like sending a response to a GetTable request
for PortInfo for every port in a large fabric is going to end up
sending a very large RMPP message, probably too large to fit in a
single kmalloc()ed buffer.  So I don't think we should require that
all send requests have a single gather entry.

    Roland> Finally, some of the MAD structures like struct
    Roland> ib_mad_private look risky to me, since kernel data might
    Roland> potentially share a cache line with DMA buffers.  See
    Roland> <http://lwn.net/Articles/2265/> for a nice writeup of the
    Roland> class of bug that might be lurking.

    Sean> This sounds like a separate issue, is that the case?

Yes.  In fact I'm not sure there's really a bug there.  It's just
something questionable that I saw while trying to find the real
problem on 440SPe.

 - R.