[ofa-general] Re: Understanding IPoIB-CM

[Michael S. Tsirkin]

> Quoting Amar Mudrankit <amar.mudrankit at gmail.com>:
> Subject: Re: Understanding IPoIB-CM
> 
> 
> 
> On 8/5/07, Michael S. Tsirkin <mst at dev.mellanox.co.il> wrote:
> 
>     > Quoting Amar Mudrankit <amar.mudrankit at gmail.com>:
>     > Subject: Understanding IPoIB-CM
>     >
>     >
>     > I have been looking at the IPoIB CM code and have some queries. I'd
>     greatly
>     > appreciate it if some one could shed some light on it.
>     >
>     > Please correct me if I am wrong, but based on my understanding of the
>     code:
>     >
>     > 1) From functions ipoib_cm_create_tx_qp and ipoib_cm_create_rx_qp, it
>     looks
>     > like for communicating with a peer, we use one QP for purely
>     >    doing TX and one QP for purely doing RX (created on receipt of a CM
>     REQ).
>     > That means for every peer we are connected to, we use two
>     >   QPs. Why not use a single QP for both TX and RX of data ? On receipt of
>     a REQ
>     > from a peer or before trying to send data to peer,  we
>     >   could possibly check if we already have a QP connection to the peer (by
>     > maintaining and looking up a QP and peer UD QPN + GID
>     >  mapping) and then just use it. Or is it not as simple as that ?
> 
>     Sure, this could work. One'd have to be sure to cover all error
>     cases though.
> 
> 
> So, there are two connections established between 2 communicating peers. 1
> connection  each for transmitting in each direction. Is it correct?

That's how the current implementation behaves, yes.

>     >   On a related note, the IPoIB CM RFC 4755, section 3.3 covers the case
>     of
>     > simultaneous IB connection where REQs can cross paths.
>     >   But the current implementation does not seem to handle this case. Is it
>     > because we create two QPs anyway and do not need to handle
>     >   this special case ?
> 
>     Yes.
> 
>     > 2] In the ipoib_cm_rep_handler, the skbs seem to be dequeued and queued
>     again.
>     > I am referring to the code below. What does this exactly
>     >    achieve ?
>     >
>     >         skb_queue_head_init(&skqueue);
>     >         spin_lock_irq(&priv->lock);
>     >         set_bit(IPOIB_FLAG_OPER_UP, &p->flags);
>     >         if (p->neigh)
>     >                 while ((skb = __skb_dequeue(&p->neigh->queue)))
>     >                         __skb_queue_tail(&skqueue, skb);
>     >         spin_unlock_irq(&priv->lock);
>     >         while ((skb = __skb_dequeue(&skqueue))) {
>     >                 skb->dev = p->dev;
>     >                 if (dev_queue_xmit(skb))
>     >                         ipoib_warn(priv, "dev_queue_xmit failed "
>     >                                    "to requeue packet\n");
>     >         }
> 
>     This transmits all skbs.
> 
> 
>         Here, I wanted to ask  - before transmitting why is the skb->dev field 
> modified as the device is same IPoIB interface through which packets will be
> transmitted. Is p->dev is some other device?

Good question, I don't recall at the moment.
I think I copied this from ipoib_main.c
Roland, any idea why do we do this?


-- 
MST