[ofa-general] RE: Problem with using two interfaces with rdma-cm

[Sundeep Narravula]

On Wed, 23 May 2007, Sean Hefty wrote:

> >Rail 1 (ib0): 192.168.1.*
> >Rail 2 (ib2): 192.168.3.*
> >
> >When I try to connect two qps over these rails (one on each), many times
> >the address resolutions for both the qps return me the context of just
> >one of the rails. i.e. I am not able to use both the rails.
> >
> >Is there any thing I am missing here?
>
> Can you provide more details on how you are establishing your connections?
>
> Are you calling rdma_resolve_addr() with 192.168.1.x in one case, and
> 192.168.3.x in the second case, and both of those resolve back to the same local
> IP address?  Can you tell if ping routes the same way?

Basically I have the following sequence of steps.

Process 1:

rdma_bind_addr (someport, 0.0.0.0)
rdma_listen ()
....
  rdma_get_cm_event()
	if (RDMA_CM_EVENT_CONNECT_REQUEST)
		rdma_accept()
....

Process 2:

rdma_connect (192.168.1.x)
rdma_connect (192.168.3.x)
....
wait for connections.


I am able to ping on both the interfaces. The ping messages go over both
the interfaces. Infact after pinging the interfaces from each other a few
times I am able to connect properly over both the rails for some time.
After a few minutes it falls back to just one interface.

> You can try binding to a specific local address, but based on your setup, I
> would expect this to work.  So, I'd like to understand better what the issue
> could be.

hmm.. I can try this but as an last resort. Ideally I would like to use
just one listen cm_id binded to 0.0.0.0.

  --Sundeep.

>
> - Sean
>