[openib-general] [ANNOUNCE] GSI Implementation Candidate

[Roland Dreier]

    Grant> Most NUMA machines are very likely to break this
    Grant> assumption.  Even IA64 on ZX1 will break the assumptions
    Grant> about contigous memory.

That map is OK... the problem would be if someone put half their RAM
at 0x80000000000000 and half at 0xc0000000000000 or something like
that.  Small holes are fine; the DMA mapping code just won't return
addresses in that area.

    Grant> I don't understand how protection domains play with DMA. I
    Grant> gather so far the HCA hosts a virtual -> DMA mapping table
    Grant> which the HCA driver keeps up to date. Is the protection
    Grant> domain used by the HCA to limit which entries in the table
    Grant> a process on a remote host may derefence?

Every queue pair and every memory region is created in some protection
domain.  For a memory region to be used by an operation on a queue
pair, their protection domains must match.  However, the verbs
extensions define a "default L_Key" (for kernel use only) that turns
off both virtual->physical translation and protection domain
checking.  It's almost possible to simulate this on the current
Mellanox HCA, except that I don't know of a way to turn off protection
domain checking, so we'll have to create one pseudo-default L_Key per
protection domain.

 - Roland


 - Roland