[openib-general] Reregister Memory Region Verb

[Caitlin Bestler]

Fab Tillier wrote:
>> From: Caitlin Bestler [mailto:caitlinb at broadcom.com]
>> Sent: Wednesday, January 25, 2006 10:06 AM
>> 
>> Fab Tillier wrote:
>>>> From: Caitlin Bestler [mailto:caitlinb at broadcom.com]
>>>> Sent: Wednesday, January 25, 2006 9:22 AM
>>>> 
>>>>>> although we would prefer that it wouldn't block if possible
>>>>> 
>>>>> mmm. All the current memory registration verbs both user and
>>>>> kernel are blocking, is it an issue for you?
>>>>> 
>>>> 
>>>> If you need to do memory registrations in a context where blocking
>>>> is not an option then you really need FMR work requests as in RDMAC
>>>> and InfiniBand 1.2 verbs.
>>> 
>>> No.  The blocking semantics of memory registration APIs is a
>>> deliberate design choice and not a limitation of the hardware.  It
>>> is possible (though more complicated) to make the API asynchronous.
>>> No existing IB stack to date has ever done so, however.
>>> 
>>> - Fab
>> 
>> If asynchronous memory registration (via work request) is such a bad
>> idea then why is it part of both the RDMAC iWARP and InfiniBand 1.2
>> verb specifications?
> 
> You misunderstood me.  I didn't say anything about FMR being
> a bad idea, just that regular MRs could be made to work in a
> non-blocking manner.  Non-blocking calls don't require FMR, it could
> be done without. 
> 
> - Fab

Yes, it is possible to specify a set of conditions where
a memory registration verb would not have to block. But
is it worthwhile to specify that under those conditions
that it MUST NOT block?

For verification purposes it is much simpler if a given
verb is either guaranteed to never block, or is considered
subject to blocking. It is much easier to check whether a
routine that is supposed to be non-blocking NEVER makes a
call to a routine that could block than it is to check that
it never makes a call to a routine with the set of conditions
that might cause it to block.

So if applications have need to do registration where they
are *guaranteed* that they will not block then I believe
an asynch API (i.e., work requests) is a much better
solution than adding lots of asterisks explaining
when a call that normally "can block" will in fact
be guaranteed not to block. The list of non-blocking
scenarios is real easy to generate as a SHOULD NOT
list, but it gets very tricky if you convert it to
a MUST NOT list.