[ofw] CM ref counting issues...

[Hefty, Sean]

>+        default:
>+            CL_ASSERT( p_mad->p_mad_buf->attr_id == CM_DREQ_ATTR_ID );
>+            /* Treat as a timeout so we don't stall the state machine. */
>+            p_mad->status = IB_WCS_TIMEOUT_RETRY_ERR;

I'm wondering about this and if it's even needed.  Changing the status isn't intuitive to someone reading the code, plus it hindered trying to debug the problem because the MAD was reported to the user with a timeout error, rather than canceled.  A canceled MAD indicates normal operation for the CM.

The mad status isn't used further down in the function to drive the state machine.  The states are driven solely based on the current state and the fact that a mad has failed.  I'm going to try removing this, so if you're aware of something that will immediately break, let me know.

- Sean