[ofa-general] spin_lock_irqsave in ib_send_mad

[Suresh Shelvapille]

Sean:

I mis-read the 'If' condition, see how the below looks:
I have a related question:
Why is send_queue.count incremented even when the wr goes on the overflow list?

BTW, I am trying to do all this because in the same thread (ib_post_send calls
Our driver post_send etc...) I have to call dev_queue_xmit which complains when
Irqs are disabled. I can get around that different ways in my driver (in a not so
elegant manner), hence all this....

----------------------------------------------------------------

        spin_lock_irqsave(&qp_info->send_queue.lock, flags);
        if (qp_info->send_queue.count < qp_info->send_queue.max_active) {
+               qp_info->send_queue.count++;

+               spin_unlock_irqrestore(&qp_info->send_queue.lock, flags);

                ret = ib_post_send(mad_agent->qp, &mad_send_wr->send_wr,
                                   &bad_send_wr);
                 
+               spin_lock_irqsave(&qp_info->send_queue.lock, flags);
                list = &qp_info->send_queue.list;
        } else {
                ret = 0;
+               qp_info->send_queue.count++;
                list = &qp_info->overflow_list;
        }

        if (!ret)
                list_add_tail(&mad_send_wr->mad_list.list, list);
+        else
+               qp_info->send_queue.count--;

---------------------------------------------------------------------


Many thanks,
Suri 

> -----Original Message-----
> From: Sean Hefty [mailto:sean.hefty at intel.com]
> Sent: Wednesday, June 10, 2009 3:07 PM
> To: 'Suresh Shelvapille'; general at lists.openfabrics.org
> Cc: 'Roland Dreier'
> Subject: RE: [ofa-general] spin_lock_irqsave in ib_send_mad
> 
> >----------------------------------------------------
> >if (qp_info->send_queue.count < qp_info->send_queue.max_active) {
> 
> This loses the locking around send_queue.count, which can allow trying to post
> more sends than the QP supports.
> 
> >                ret = ib_post_send(mad_agent->qp, &mad_send_wr->send_wr,
> >                                   &bad_send_wr);
> >                spin_lock_irqsave(&qp_info->send_queue.lock, flags);
> >                list = &qp_info->send_queue.list;
> >} else {
> >                ret = 0;
> >                spin_lock_irqsave(&qp_info->send_queue.lock, flags);
> >                list = &qp_info->overflow_list;
> >}
> 
> I think the code will end up quite a bit more complex not locking around
> ib_post_send, but maybe there's a clever way to handle this.
> 
> - Sean