[ofw] Name resolution in WinVerbs/Multiple cards in one machine

[Smith, Stan]

When you set the Local Area Connection (IPoIB) DHCP or static IPv4 address, this IP address is bound to the HCA/port pair.
See ncpa.cpl->LAC(IPoIB) properties along with ipconfig.

Stan.

From: Christoph Müller [mailto:christoph.mueller at visus.uni-stuttgart.de]
Sent: Wednesday, January 25, 2012 1:24 PM
To: Hefty, Sean; ofw_list
Cc: Smith, Stan
Subject: AW: Name resolution in WinVerbs/Multiple cards in one machine

Hi Sean,

thanks for your explanations. I already thought of some kind of DNS-related problem and will try to fix that tomorrow. In the meantime, I have another question: Is it sufficient if the name server returns the IPoIB address at any position in the host entry? It would be a problem for me if it had to be the first. And can the machines be in different sub domains and nets? I do still not understand how the IP/GID mapping of a remote port is done in WV - I would expect the need for some kind of out of band mechanism here, too.

Best regards,
Christoph

Gesendet von meinem Windows Phone
________________________________
Von: Hefty, Sean
Gesendet: 25.01.2012 20:27
An: Christoph Müller; ofw_list
Cc: Smith, Stan
Betreff: RE: Name resolution in WinVerbs/Multiple cards in one machine
This message was forwarded to me, since it apparently never made it to the mail list.

> I have some problems understanding how the name resolution for IB devices
> works. My primary problem is that my WinVerbs-based application ceased
> functioning because OpenDevice() fails - it has worked before though, and I am
> not aware of any configuration changes so far. The problem seems to be that
> the device GUID that I use is invalid. I obtain this GUID from a Query() on
> the connect endpoint after this end point was bound, but the GUID is all null.
> I am pretty sure that it has not been before...

Name resolution depends on converting the hostname into an IP address using DNS.  There's nothing special that the IB stack provides to make this happen.  IP resolution requires mapping the IP address to an IB address (typically GID).  For this to work, ipoib is required.  A lookup is done to see what IB GID the IP address is associated with.

Note that native IB devices do not know of or use IP addresses.  The IP to IB address mapping is simply a convenience provided by the IB stack.  So, to map from a hostname to an IB address, DNS is used to convert the hostname into an IP address, and then ipoib is used to map that IP address into a GID.

> I assume that the subnet manager holds mapping from machine names to GUIDs, is
> that correct?

No - the IB subnet manager only knows of GIDs, LIDs, and GUIDs.

> I further assume that the subnet manager uses TCP/IP, DNS,
> whatever to obtain this information, and that might be the problem.

The local host software will handle this mapping.

> A further question would be: If I know the port GUID, is there any possibility
> to use it directly with WinVerbs? I can only see a Connect() accepting a
> socket address...

You can, but you would need to establish the connection using some sort of out of band mechanism (such as sockets) and manually configure the QPs.  If you have the source code, you may want to look at libibverbs and those samples.  libibverbs sits over winverbs and the samples will establish connections out of band.

> And is there any documentation on how to use multiple IB devices (dual port)
> in one machine? I have not found any way to address the ports separately so
> far, because I do not understand how the mapping of the socket address to the
> port GUID works.

Assuming that you get name resolution working again, as long as each port is assigned a different IP address you should be able to address each port separately.  You may need to use the IP address for your resolution rather than the host name.

- Sean
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofw/attachments/20120125/b68be0e5/attachment.html>