<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal">Does anyone know of a mailing list specific for soft-roce / rxe?<o:p></o:p></p>
<p class="MsoNormal">I’ve been trying to get the VMs as described in the OFA workshop “Building a Virtual Cluster” tutorial from 2019 up and running.<o:p></o:p></p>
<p class="MsoNormal">I’m not sure if I have it correct yet. There were a few differences due to newer versions of the virtualbox, but I am fairly sure I followed the tutorial closely.<o:p></o:p></p>
<p class="MsoNormal">The results of the ibv_devices, ibv_devinfo, and the rxe_cfg, lsmod for the rxe and ib_uverbs all look ok.<o:p></o:p></p>
<p class="MsoNormal">I’m getting the qperf to run with the –cm1 option.<o:p></o:p></p>
<p class="MsoNormal">I can get rping to run with slightly different command lines than on the rxe_dev wiki (ref.
<a href="https://github.com/SoftRoCE/rxe-dev/wiki/Validate-that-RXE-is-working">https://github.com/SoftRoCE/rxe-dev/wiki/Validate-that-RXE-is-working</a>), but it at least works.<o:p></o:p></p>
<p class="MsoNormal">But I cannot seem to get the ibv_rc_pingpong to run.<o:p></o:p></p>
<p class="MsoNormal">There doesn’t appear to have been much done on the github site for a long time.<o:p></o:p></p>
<p class="MsoNormal">I’m a bit stuck. I don’t have any live hardware yet. I was hoping to at least get a hello world rdma transfer going using libibverbs before then, but now I don’t know if I can even trust this setup. I’ll have to go down that path a ways
blindly. <o:p></o:p></p>
<p class="MsoNormal">I’m using the distro versions for everything. I’m not compiling soft-roce myself (or the kernel).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The VMs are centos 7.6 boxes.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">On the server<o:p></o:p></p>
<p class="MsoNormal">ibv_rc_pingpong –g 1 –d rxe0 –I 1<o:p></o:p></p>
<p class="MsoNormal">gives:<o:p></o:p></p>
<p class="MsoNormal">local address: LID 0x0000, QPM 0x000011, PSN 0xcce8ae, GIC ::ffff:192.168.56.6<o:p></o:p></p>
<p class="MsoNormal">Failed to modify QP to RTR<o:p></o:p></p>
<p class="MsoNormal">Couldn’t connect to report QP<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">On the client:<o:p></o:p></p>
<p class="MsoNormal">ibv_uc_pingpong –d rxe0 –g 1 compute_node-1-roce<o:p></o:p></p>
<p class="MsoNormal">local address: LID 0x0000, QPM 0x000011, PSM 0xcaa6e0, GID fe80::538b:f720:3a3b:b0a7<o:p></o:p></p>
<p class="MsoNormal">client read/write: Success<o:p></o:p></p>
<p class="MsoNormal">Couldn’t read/write remote address<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Anyone have any helpful suggestions?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks<o:p></o:p></p>
<p class="MsoNormal">Andy<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
------------------------------------------- CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient and may contain material that is proprietary, confidential, privileged or otherwise legally protected or restricted
under applicable government laws. Any review, disclosure, distributing or other use without expressed permission of the sender is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies without reading, printing,
or saving..
</body>
</html>