[openib-general] problem using two network interfaces on the same HCA

[Leonid Arsh]

Hello,

I have a problem using different IPoIB network interfaces on the same
HCA with two ports.

I have two hosts with two port HCAs
Both HCAs are connected to a switch with both their ports.

I configure ib0 and ib1 interfaces to with different subnet IPs, just
like
   ib0 192.168.3.235, ib1 192.168.4.235 on the first machine
and
   ib0 192.168.3.156, ib1 192.168.4.156 on the second one.
Pings work only via one of the interfaces. When I begin sending pings
via second one, the first interface stops receiving replies.
Example:
   ping 192.168.3.156 - OK
   ping 192.168.4.156 - OK
   ping 192.168.3.156 again - no reply.



I tried to configure the network interfaces on different IB partitions,
like
  ib0 192.168.3.235,   ib1.f1f1 192.168.4.235 on the first machine
and
   ib0 192.168.3.156,  ib1.f1f1  192.168.4.156 on the second one.
It worked fine.
 
 
I also tried to run the first test (the same partition),
using two HCAs on the same machine, and connecting one port on every HCA
to the switch.
(   ib0 192.168.3.235, ib1 192.168.4.235 on the first machine
   ib0 192.168.3.156, ib2 192.168.4.156 on the second one, where ib2 is
the interface on the second HCA )
It worked fine too.
 
 
You could say that configuring different IP subnets on the same physical
network isn't a good practice, and the problem is in the TCP/IP stack,
but
two network interfaces on different HCAs worked fine, while
two network interfaces on different ports on the same HCA made problems.
 
Has anyone seen a problem like this?
Any ideas?
 
I run 
SUSE Linux Enterprise Server 9.90 Beta5  distribution,
with kernel 2.5.15 installed,,
and IB kernel modules from OpenIB SVN, revision 5337
on dual CPU x86_64 machines.
 
Leonid Arsh
SW Engineer
www.voltaire.com <http://www.voltaire.com/> 
The Grid Interconnect Company

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20060308/863dd859/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Blank Bkgrd.gif
Type: image/gif
Size: 145 bytes
Desc: Blank Bkgrd.gif
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20060308/863dd859/attachment.gif>