[libfabric-users] netdir / verbs interop

[smoore at MRU.MEDICAL.CANON]

Hi Sean,

Thanks for the quick response!

>>> Interopt between Linux and Windows has never been tested.  It should be able to be made to work, but likely requires work.
I stumbled across one interop issue already.  netdir supports up to 256 bytes of CM_DATA, verbs only supports 56 bytes (VRB_CM_DATA_SIZE), and verbs sets the first byte to the length of the data.  fi_setopt isn't implemented for either provider.  Is there any other way to change this?  Again, I suspect not, just want to confirm.

I'll let you know how it goes when I get further.

Best Regards,

Steve

-----Original Message-----
From: Hefty, Sean <sean.hefty at intel.com> 
Sent: Thursday, November 12, 2020 15:30
To: Moore, Steve <smoore at MRU.MEDICAL.CANON>; libfabric-users at lists.openfabrics.org
Subject: RE: netdir / verbs interop

> I must transfer data via RoCE between Linux and Windows using a 
> SmartNIC to offload the CPU and bypass the OS kernel.  Both Linux and 
> Windows machines will be equipped with Mellanox ConnectX-5 NICs.

Interopt between Linux and Windows has never been tested.  It should be able to be made to work, but likely requires work.

> My basic question is what libfabric providers must I use?  From 
> reading the docs, I think I must use the verbs provider on Linux and 
> the netdir provider on Windows.  Is this correct?

This would be the correct combination.

> As a follow up, is the verbs provider supported on Windows?  I suspect 
> not, but would like to confirm.

No - there are not libibverbs/librdmacm interfaces available on Windows.

- Sean

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________