<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><br></div>Has anyone tested the collectives offload provided by MLNX OFED?<div><br></div><div>I have everything set up correctly according to the FCA documentation and fca_managerd is running.</div><div>My attempts to run with "coll_fca_enable 1" are failing.</div><div>First with an inability to talk to umad port</div><div><br></div><div><div>ibwarn: [2337] mad_rpc_open_port: can't open UMAD port (mlx4_0:1)</div><div>1345251776.473792 [FCA_DEV cja001 2337] dev.c:560 error Failed to initialize SA: Cannot assign requested address</div><div><br></div><div>The permissions on /dev/infiniband/umad0 were opened up - which got rid of the error messages.</div><div>Now it is dying and spitting out a bunch of empty core files - but nothing in the job run log. </div><div>It looks like it is going to start, then poof !</div><div><br><div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>====================================</div><div><br></div><div>Susan Coulter<br>HPC-3 Network/Infrastructure<br>505-667-8425<br>Increase the Peace...<br>An eye for an eye leaves the whole world blind<br>====================================</div></div></span></div>
</div>
<br></div></div></body></html>