[ofa-general] spin_lock_irqsave in ib_send_mad

[Sean Hefty]

>        spin_lock_irqsave(&qp_info->send_queue.lock, flags);
>        if (qp_info->send_queue.count < qp_info->send_queue.max_active) {
>+               qp_info->send_queue.count++;
>
>+               spin_unlock_irqrestore(&qp_info->send_queue.lock, flags);
>
>                ret = ib_post_send(mad_agent->qp, &mad_send_wr->send_wr,
>                                   &bad_send_wr);
>
>+               spin_lock_irqsave(&qp_info->send_queue.lock, flags);
>                list = &qp_info->send_queue.list;
>        } else {
>                ret = 0;
>+               qp_info->send_queue.count++;
>                list = &qp_info->overflow_list;
>        }
>
>        if (!ret)
>                list_add_tail(&mad_send_wr->mad_list.list, list);
>+        else
>+               qp_info->send_queue.count--;

It's not quite this simple.  Once the lock is released before calling
ib_post_send, another thread could come down and queue a MAD to the overflow
list.  If ib_post_send fails, the overflow list must be checked to see if a
queued mad should now be sent.

As for being able to hold a lock when calling ib_post_send, that's something
that should be allowed.

- Sean