[Users] soft-roce configuration?

Tue May 21 14:35:46 PDT 2019

Does anyone know of a mailing list specific for soft-roce / rxe?
I've been trying to get the VMs as described in the OFA workshop "Building a Virtual Cluster" tutorial from 2019 up and running.
I'm not sure if I have it correct yet. There were a few differences due to newer versions of the virtualbox, but I am fairly sure I followed the tutorial closely.
The results of the ibv_devices, ibv_devinfo, and the rxe_cfg, lsmod for the rxe and ib_uverbs all look ok.
I'm getting the qperf to run with the -cm1 option.
I can get rping to run with slightly different command lines than on the rxe_dev wiki (ref. https://github.com/SoftRoCE/rxe-dev/wiki/Validate-that-RXE-is-working), but it at least works.
But I cannot seem to get the ibv_rc_pingpong to run.
There doesn't appear to have been much done on the github site for a long time.
I'm a bit stuck. I don't have any live hardware yet. I was hoping to at least get a hello world rdma transfer going using libibverbs before then, but now I don't know if I can even trust this setup. I'll have to go down that path a ways blindly.
I'm using the distro versions for everything. I'm not compiling soft-roce myself (or the kernel).

The VMs are centos 7.6 boxes.

On the server
ibv_rc_pingpong -g 1 -d rxe0 -I 1
local address: LID 0x0000, QPM 0x000011, PSN 0xcce8ae, GIC ::ffff:
Failed to modify QP to RTR
Couldn't connect to report QP

On the client:
ibv_uc_pingpong -d rxe0 -g 1 compute_node-1-roce
local address: LID 0x0000, QPM 0x000011, PSM 0xcaa6e0, GID fe80::538b:f720:3a3b:b0a7
client read/write: Success
Couldn't read/write remote address

Anyone have any helpful suggestions?


------------------------------------------- CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient and may contain material that is proprietary, confidential, privileged or otherwise legally protected or restricted under applicable government laws. Any review, disclosure, distributing or other use without expressed permission of the sender is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies without reading, printing, or saving..
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20190521/c5f40251/attachment.html>

More information about the Users mailing list