[libfabric-users] Question about FI_THREAD_SAFE and other threading models...

Heinz, Michael William michael.william.heinz at cornelisnetworks.com
Mon May 3 09:46:08 PDT 2021

Got it, thanks.

-----Original Message-----
From: Hefty, Sean <sean.hefty at intel.com> 
Sent: Monday, May 3, 2021 12:36 PM
To: Heinz, Michael William <michael.william.heinz at cornelisnetworks.com>; libfabric-users at lists.openfabrics.org
Subject: RE: Question about FI_THREAD_SAFE and other threading models...

> The definition of FI_THREAD_SAFE in the fi_domain(3) man page says:
> 	A thread safe serialization model allows a multi-threaded application 
> to access any allocated resources through any interface without 
> restriction. All providers are required to support FI_THREAD_SAFE.
> That appears to mean that all providers are required to handle any 
> serialization/locking that might be required and that the application 
> can be as multi- threaded as desired...?

Correct -- that is the intent.

> If that's the case, then why do we bother with more restrictive 
> serialization models like FI_THREAD_ENDPOINT? Wouldn't they be redundant?

If the application is single threaded or assigns fabric resources to threads (no sharing), providers can eliminate locks.  The app indicates this through the threading attribute.  Some of the providers do this internally.  As an example, the base endpoint implementation does not provide locking unless the app requires thread safety:

	if (ep->domain->threading != FI_THREAD_SAFE) {
		ep->lock_acquire = ofi_fastlock_acquire_noop;
		ep->lock_release = ofi_fastlock_release_noop;
	} else {
		ep->lock_acquire = ofi_fastlock_acquire;
		ep->lock_release = ofi_fastlock_release;

The base completion queue implementation is similar, but more restrictive as to when locks can be removed:

	if (cq->domain->threading == FI_THREAD_COMPLETION ||
	    (cq->domain->threading == FI_THREAD_DOMAIN)) {
		cq->cq_fastlock_acquire = ofi_fastlock_acquire_noop;
		cq->cq_fastlock_release = ofi_fastlock_release_noop;
	} else {
		cq->cq_fastlock_acquire = ofi_fastlock_acquire;
		cq->cq_fastlock_release = ofi_fastlock_release;

- Sean

More information about the Libfabric-users mailing list