[openib-general] basic IB doubt

[Talpey, Thomas]

At 09:00 AM 8/28/2006, Gleb Natapov wrote:
>> 2) user must deregister any mapping before inspecting the result. I
>> doubt any of them do this, for that reason anyway.
>> 
>This may have big performance impact.

You think? :-)

>> MO is that this will bite us in the a** some day. If anybody was
>> running this code on the Sparc architecture it already would have.
>> 
>AFAIK SUN runs MPI over UDAPL, but they have their own IB
>implementation, so may be they handle all coherency issues in the UDAPL
>itself.

The Sparc IOMMU supports consistent mappings, in which the
i/o streaming caches are not used. There is a performance
impact to using this mode however. The best throughput is
achieved using streaming with explicit software consistency.

However, even in consistent mode, the Sparc API requires
that the synchronization calls be made. I have never gotten
a completely satisfactory answer as to why, but on the
high-end server platforms, I think it's possible that the busses
can't always snoop one another and the calls provide a "push".

Will turning on the Opteron's IOMMU introduce some of these
issues to x86?

Tom.