<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Aptos;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Aptos",sans-serif;
mso-ligatures:standardcontextual;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:11.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1120339295;
mso-list-type:hybrid;
mso-list-template-ids:882829902 998545712 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Aptos",sans-serif;
mso-fareast-font-family:Aptos;
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1
{mso-list-id:1272669220;
mso-list-type:hybrid;
mso-list-template-ids:-160917820 638084098 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l1:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Aptos",sans-serif;
mso-fareast-font-family:Aptos;
mso-bidi-font-family:"Times New Roman";}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#467886" vlink="#96607D" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">04/01/2025<o:p></o:p></p>
<p class="MsoNormal"><b><u>Participants:<o:p></o:p></u></b></p>
<p class="MsoNormal">Alexia Ingerson (Intel)<o:p></o:p></p>
<p class="MsoNormal">Jianxin Xiong (Intel)<o:p></o:p></p>
<p class="MsoNormal">Alex McKinley (Intel)<o:p></o:p></p>
<p class="MsoNormal">Ben Lynam (Cornelis)<o:p></o:p></p>
<p class="MsoNormal">Charles Shereda (Cornelis)<o:p></o:p></p>
<p class="MsoNormal">Call-in User_1<o:p></o:p></p>
<p class="MsoNormal">Howard Pritchard (LANL)<o:p></o:p></p>
<p class="MsoNormal">Ian Ziemba (HPE)<o:p></o:p></p>
<p class="MsoNormal">Jerome Soumagne (HPE)<o:p></o:p></p>
<p class="MsoNormal">John Byrne (HPE)<o:p></o:p></p>
<p class="MsoNormal">Juee Desai (Intel)<o:p></o:p></p>
<p class="MsoNormal">Ken Raffenetti (ANL)<o:p></o:p></p>
<p class="MsoNormal">Peinan Zhang (Intel)<o:p></o:p></p>
<p class="MsoNormal">Rajalaxmi Angadi (Intel)<o:p></o:p></p>
<p class="MsoNormal">Stephen Oost (Intel)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b><u>Summary:<o:p></o:p></u></b></p>
<p class="MsoNormal">2.2.0 release targeted for 6/15 – have your big patches in by the end of May. Big features changes for this release are a new shm architecture and a refactoring of the lnx provider<o:p></o:p></p>
<p class="MsoNormal">Bug scrub: went over newer issues to make sure they are getting addressed.<o:p></o:p></p>
<p class="MsoNormal">Discussion regarding setting of environment variables for different client instances within the same process (for example MPI and DAOS). Suggested using domain ops to specify options for different instances.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b><u>Notes:<o:p></o:p></u></b></p>
<p class="MsoNormal">Planning next release (2.2.0 )<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l1 level1 lfo1">RC1 6/1<o:p></o:p></li><li class="MsoNormal" style="mso-list:l1 level1 lfo1">GA 6/15 – big patches by end of May<o:p></o:p></li><li class="MsoNormal" style="mso-list:l1 level1 lfo1">Big features: new shm, new lnx<o:p></o:p></li></ul>
<p class="MsoNormal">Going over open issues:<o:p></o:p></p>
<p class="MsoNormal">#10911: Verbs CSWAP fetch result error<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Able to reproduce with OFI. Looks like endianess issue or byte placement issue. Will look into it<o:p></o:p></li></ul>
<p class="MsoNormal">#10887: LINKx MPI_Probe segfault<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Waiting for new link architecture to address since code will be so different. Will be fixed in new linkx<o:p></o:p></li></ul>
<p class="MsoNormal">#10881: Reload verbs devices on each getinfo call<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Being addressed in PR<o:p></o:p></li></ul>
<p class="MsoNormal">#10880: Possible to specify source port on libfabric RDMA client side<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Anyone have any experience?<o:p></o:p></li></ul>
<p class="MsoNormal">#10879: deadlock with mimalloc<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Rbmap insert gets called with mm_lock held. Madvise calls insert callback again leading to deadlock<o:p></o:p></li></ul>
<p class="MsoNormal">#10865: OSU segfault on linkx with cxi cuda<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">HPE taking a look<o:p></o:p></li></ul>
<p class="MsoNormal">#10860: Build error on Perlmutter<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">System specific - HPE will take a look<o:p></o:p></li></ul>
<p class="MsoNormal">#10852: verbs async events<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">PR under review<o:p></o:p></li></ul>
<p class="MsoNormal">#10847: missing rxm CQ entry flags<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Original issue fixed but maybe other issue exists, waiting on reply from reporter<o:p></o:p></li></ul>
<p class="MsoNormal">#10822: cxi low performance<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">HPE will take a look<o:p></o:p></li></ul>
<p class="MsoNormal">#10821: shm prov key support<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Not really a bug/workaround in util mr map may fix, refactor may fix<o:p></o:p></li></ul>
<p class="MsoNormal">#10823: efa control plan AV operation locking<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Efa issue<o:p></o:p></li></ul>
<p class="MsoNormal">#10804: unsafe reads of av_entry_pool because no lock<o:p></o:p></p>
<p class="MsoNormal">#10798: improve rdm ep for storage<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Continued discussion<o:p></o:p></li></ul>
<p class="MsoNormal">#10785: tcp debug build generates lock in use assert<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Client is using two threads – but threading model is set to FI_THREAD_DOMAIN so locks are getting set to noops<o:p></o:p></li><li class="MsoNormal" style="mso-list:l0 level1 lfo2">Will look into threading implementation<o:p></o:p></li></ul>
<p class="MsoNormal">#10762: data race in ibv_req_notify_cq<o:p></o:p></p>
<p class="MsoNormal">#10692: device max_cqe not used to set CQ size<o:p></o:p></p>
<p class="MsoNormal">#10589: psm3 illegal instruction<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Fixed<o:p></o:p></li></ul>
<p class="MsoNormal">#10566: calling fi_connect and fi_eq_sread costs a lot of time<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">System specific, no response, closing<o:p></o:p></li></ul>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Discussion: Setting libfabric environment variables per client in a process<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">Separate libraries in user process want to both use OFI with different environment variables<o:p></o:p></li><li class="MsoNormal" style="mso-list:l0 level1 lfo2">Could have domain specific op<o:p></o:p></li><li class="MsoNormal" style="mso-list:l0 level1 lfo2">Related to issue #10526 (add runtime setting to select MR cache monitor)<o:p></o:p></li><li class="MsoNormal" style="mso-list:l0 level1 lfo2">Domain option should be reasonable since client instances will be using different domains<o:p></o:p></li></ul>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>