OK, this works on my i386 system but I'm still getting ib_mad: Invalid directed route on ppc64. I'll try to debug what exactly is happening (ie put some prints in to see why smi_handle_dr_smp_send() is rejecting it). - R.