<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<style>
<!--
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:Arial;}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:Arial;
color:windowtext;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
/* List Definitions */
@list l0
{mso-list-id:305162995;
mso-list-type:hybrid;
mso-list-template-ids:-1980349586 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:.75in;
mso-level-number-position:left;
margin-left:.75in;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
-->
</style>
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'>Roland,<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'>I am
trying to write a user level application that receives multicast UD packets at
user level. I am seeing about 1-2 % packet loss between the send side and the
receive side apparently independent of the packet rate for low rates. (Heavily
traced sends and receives with very low rates still drop packets even though
there are more packets posted on the receive side than are sent.) I have a
couple of questions:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal style='margin-left:.75in;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><font
size=2 face=Arial><span style='font-size:10.0pt'><span style='mso-list:Ignore'>1.<font
size=1 face="Times New Roman"><span style='font:7.0pt "Times New Roman"'>
</span></font></span></span></font><![endif]><font size=2><span
style='font-size:10.0pt'>Are there any race issues with ibv_get_cq_event? The
example code (ud_pingpong) seems to imply that the correct sequence is<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal style='margin-left:1.0in'><font size=2 face=Arial><span
style='font-size:10.0pt'>Start:<o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:1.0in'><font size=2 face=Arial><span
style='font-size:10.0pt'> Call
ibv_get_cq_event<o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:1.0in'><font size=2 face=Arial><span
style='font-size:10.0pt'> Call
ibv_ack_cq_event <-
anywhere so long as it happens before destroy_cq<o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:1.0in'><font size=2 face=Arial><span
style='font-size:10.0pt'> Call
ibv_req_notify_cq<o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:1.0in'><font size=2 face=Arial><span
style='font-size:10.0pt'> Call
ibv_poll_cq <-
just once not as usual until empty according to the example<o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:1.0in'><font size=2 face=Arial><span
style='font-size:10.0pt'> Goto
start<o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:1.0in'><font size=2 face=Arial><span
style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 face=Arial><span
style='font-size:10.0pt'>In the old days we called request notify and poll
until poll was empty on a notify thread in order to prevent a race.<o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 face=Arial><span
style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal style='margin-left:.75in;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><font
size=2 face=Arial><span style='font-size:10.0pt'><span style='mso-list:Ignore'>2.<font
size=1 face="Times New Roman"><span style='font:7.0pt "Times New Roman"'>
</span></font></span></span></font><![endif]><font size=2><span
style='font-size:10.0pt'>When I post say 500 receive buffers and send say 200 send
buffers and tag the sends with a sequence number I often see one or two missing
sequence numbers at the receive side at the poll_cq interface having checked at
the post and poll interfaces of the send side to see that all the correct
sequence numbers went out. I am not sure how this can be possible regardless of
the notification scheme used.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'>I
would love for this to be a programming error in my code but I can’t
figure out how I can mess it up between post_send and poll_cq on the receive
side. I see the same behavior between systems and with a loopback between two
ports on the same HCA.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'>Please
let me know if this rings any bells.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'>Bob
Pearson<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt'><o:p> </o:p></span></font></p>
</div>
</body>
</html>