[openib-general] Reserved L_Key API

[Roland Dreier]

    Roland> On the other hand even there I can think of hacks to get
    Roland> around the problem -- the only systems where this
    Roland> limitation really causes problems would be Intel IA-32 and
    Roland> EMT64T systems with > 4 GB of RAM (since Intel has no
    Roland> IOMMU).  And for those systems the assumption that RAM
    Roland> starts at PCI address 0 is valid, so you could just
    Roland> register a region from 0 up to 0xfffffffff (ie the low 64
    Roland> GB) and be happy.

    Dror> Not sure I understand. If we do plan on supporting
    Dror> architectures that can't map the whole address space to DMA,
    Dror> then I think that the API should reflect it. When you try to
    Dror> create a DMA MR, then you should be able to request which is
    Dror> the address space that you need. Am I missing something ?

Yes, you're looking at things backwards.  The device knows what its
DMA mask is so there's no problem if an architecture limits it to DMA
only in the low 4G.  There's no need to put this in the API because in
fact it's the consumer that doesn't know the limits on DMA addresses.

The issue is a device that cannot create a MR that spans the full 64
bit address space (as I said, based on my quick look at the
InfiniCon/Agilent driver, the Fujitsu HCA seems to be in this
category).  For systems with an IOMMU, the driver can just set its DMA
mask to the low 4G and be happy.  For systems without IOMMU (ie Intel
IA32/EMT64) the driver can set its DMA mask to the full 64 bits and
create an MR spanning the low 64G and still be happy.

 - Roland