[ofw] [RFC] [PATCH 1/2] work_queue: new abstraction for queuing work

[Sean Hefty]

>While IO_WORKITEMs can't be embedded in structures pre-Vista, you can allocate
>a work item and store the pointer to it pre-Vista.

Long story short, I think a WORK_ENTRY type structure is still needed.

The IO_WORKITEM is associated with a WDFREQUEST.  A structure is still needed to
wrap around the WDFREQUEST and the IO_WORKITEM, because the WorkItem() callback
only provides a single context pre-Vista.  (Any work item 'Ex' functions are
essentially unusable for our purposes, as they require Vista.)

The processing of the WDFREQUEST in the system thread provides the necessary
synchronization to handle object destruction and device removal.  This involves
acquiring and releasing the WV_ENDPOINT or WV_QUEUE_PAIR, which must be done at
passive level.  The same checks cannot be done from the CM callback at dispatch
without completely reworking all of the locking.  And because of device removal
handling, I'm not even sure that's doable.

By the time the CM callback is invoked, the EP and/or QP could be in the process
of being destroyed.  I want to allow this because of how long it could take
before the CM callback fires in the case of a DREQ timeout, which could be on
the order of a couple of minutes.

>Just store the work items (by reference) in the objects that need them.

This requires taking the reference on the object during the Disconnect down
call.  Any attempt to destroy the endpoint or a handle device removal would need
to delay until a DREP was received or the DREQ timed out.  I don't like that the
disconnect processing ends up taking a reference on the EP asynchronously at
all, since it can result in EP destroy failing.

I will see if I can extend the EP connect state information to simplify things,
while still avoiding holding a reference on the EP while the disconnect is in
progress.

>The IO_WORKITEM documentation warns against infinite loops in a work item
>handler.  See the section "System Worker Threads" in the WDK docs, the last
>paragraph: "If one of these routines runs for too long (if it contains an
>indefinite loop, for example), the system can deadlock."

>From the WDK: KMDF documentation "Using Framework Work Items":

	Creating and Deleting Work Items
	Drivers can use one of the following two techniques to
	create and delete work items:

	* Use each work item once: create the work item when you need it
	  and delete it immediately after it is used.  This technique is
	  useful for drivers that require infrequent use (less often than
	  once per minute) of a small number of work items...

	* Create one or more work items that your driver requeues as
	  necessary. This technique is useful for drivers that use work
	  items frequently (more often than once per minute), or if your
	  driver's EvtInterruptDpc callback function cannot easily handle
	  a STATUS_INSUFFICIENT_RESOURCES return value from WdfWorkItemCreate.

See WdfWorkItemCreate as a starting point and follow the links from that.  I'm
guessing the Wdf* are simply wrappers around the IO work items calls.  I didn't
use the Wdf* calls, because the work item ones looked simpler and more efficient
to me.

So, the work queue could limit the number of work items that it processes at one
time and requeue a work item, but based on the above using a single work item
(whether a WDFWORKITEM or IO_WORKITEM) per EP should be avoided.

- Sean