[openib-general] [PATCH] librdmacm: updated librdmacm to work with proposed 2.6.20 kernel CMA

[Or Gerlitz]

Sean Hefty wrote:
> Sean Hefty wrote:
>>> One of the systems kernel is actually 2.6.19-rc3 and patches 1-7 (ie 
>>> not roland's tree) and i see there some issues also with ip multicast 
>>> over ipoib. I will move to use the same kernel config (roland's tree 
>>> and patches 1-7), then test ipoib and only then mckey, will let you 
>>> know.
>>>
>>> Will you have the chance to test ipoib multicast and mckey over this 
>>> config at your environment?
> 
> This seemed to work fine for me in loopback mode.  I'm updating another 
> test system to check between systems now.

OK, ip multicast through ipoib works fine in my config, rping also works 
fine. However, mckey does not work. The receiver never polls anything 
from its cq.

Adding some debug prints at librdmacm and mckey everything seems to be 
fine (both sender and receiver use same/correct mgid/mlid, the sender 
when creating its ah and the receiver when joining etc).

The only thing i found was that the remote_qpn used for post sending was 
not 0xffffff but fixing this did not help.

Below are some prints from the receiver, the sender and the SM, where i 
can see that both ports joined the same/correct mgid.

Please let me know if you manage to get mckey working in non loopback 
mode and if yes, if you have an idea how can i further debug my config.

thanks,

Or.

> root at excell01 librdmacm]#  /home/ogerlitz/ib1.1/bin/mckey -m 224.5.5.5
> mckey: starting server
> mckey: joining
> rdma_create_qp: created qp 0x505b00
> ucma_process_join attaching to mlid c008 mgid ff12:4001:ffff:0000:0000:0000:0005:0505
> ucma_process_join: calling ibv_attach_mcast qp 0x505b00
> rdma_get_cm_event: ucma_process_join returned 0
> join handler, params: dlid   c008
> join handler, params: qp_num ffffff
> receiving data transfers
> 
> [root at excell02 src]# /home/ogerlitz/ib1.1/bin/mckey -m 224.5.5.5 -s -C 10240 -S 1024
> mckey: starting client
> mckey: joining
> rdma_create_qp: created qp 0x503c80
> ucma_process_join attaching to mlid c008 mgid ff12:4001:ffff:0000:0000:0000:0005:0505
> ucma_process_join: calling ibv_attach_mcast qp 0x503c80
> rdma_get_cm_event: ucma_process_join returned 0
> join handler, params: dlid   c008
> join handler, params: qp_num ffffff
> initiating data transfers
> data transfers complete
> test complete
> return status 0


SM prints

> MCG MLID 0xC008 MGID 0xff124001ffff00000000000000050505
> 0x0008f104039708dd  1 0x0003 0x0008f104039708dc 0008f104039708dc                 0.0.0.0
> 0x0008f104039708fd  1 0x0007 0x0008f104039708fc 0008f104039708fc                 0.0.0.0