[ofiwg] wait sets vs poll sets vs application

[Hefty, Sean]

> However, the subtle differences in the interfaces seems to make it very
> difficult to achieve kind of a consistent usage approach.  In particular:
> 
> 1) wait sets don't return context

There's been some discussion on this.  The main issue is that returning a context imposes additional implementation requirements that may not be easily met.  The wait set becomes more than just a wait object that gets signaled.  It now needs to provide a queue of what object(s) signaled the wait object, as there could have been more than one which caused the wake-up.

> 2) poll sets don't support timeouts

Poll sets are non-blocking.  A poll set allows an application to easily drive progress across multiple event/completion queues.  The poll set also checks for events across multiple queues.

> 3) poll sets don't support access to their underlying "wait" object

It sounds like you may want to try using the wait sets and poll sets together.  For example, group all objects into a wait set and a poll set.  When wait is signaled, call the poll set to get the objects that have events.  This would give you the contexts that you're wanting.

> So my question is, is the design of these object interfaces intentional in
> this regard?  Should apps just pick one approach and work with what's
> there?

The design is intentional, but that doesn't mean that the use cases are fully worked out. :)  They are designed to be used together.  Your feedback on this is definitely useful.

> I also went ahead and put epoll-based wait and poll implementations into
> my fork of the verbs provider and it seems of value (disregarding issues
> above) - though it doesn't really make sense there as its not verbs
> specific.  Perhaps any provider that uses fd's for internal waits can
> leverage a common epoll implementation?

I had the same thought as well.  And I agree that the internal implementation of wait sets could be improved by using epoll. (Well, technically, the internal implementation could be improved by being written, but assuming that it was written, then epoll seems like a nice option. :)

- Sean