[ofa-general] uDAPL Question

[Caitlin Bestler]

Don.Kerr at Sun.COM wrote:
> I am working on a uDAPL layer for Open MPI.  The situation is
> if I have more than one port/HCA my users may want to be
> selective in what is used and to do this they would need to
> provide some information regarding which port/HCA to use. So
> my thought is that the users are more familar with the output
> from "ifconfig", for example ib0, ib1, etc, and I was trying
> to find a way to correlate that to what is available from the
> uDAPL API. Maybe I need to reprogram them to look at dat.conf.
> 
> -DON
> 

You definitely do not want to parse dat.conf, you want to see
what the dat_registry has loaded. dat.conf is static, Providers
are allowed to dynamically adapt how they register themselves.
I don't believe that is an active concern, but it's simpler to
take advantage of the existing code and be safe in case somebody
comes along later and decides to do dynamic registration only.

But you hit the nail on the head in terms of needing to correlate
devices as reported by "ifconfig" and the Interface Adapter that
you try to open.

Basically, the intent has always been that the correlation between
an Interface Adapter and an "ifconfig" entry should be so obvious
that a complete idiot could figure out which went with which.
Once that linkage is clear then you merely use the RDMA device/port
implied by the routing of the device listed by ifconfig.

To the best of my knowledge, for every DAPL provider ever created
the correlation with the IP layer device has indeed been so obvious
that any idiot could figure it out -- unfortuantely software can only
hope to someday reach that degree of intelligence, and other than 
configuring the links there really isn't much that can be done.

Once there is a link between the RDMA device and the IP layer device,
you could use the routing tables to determine which port a connection
request could be received on, which ports could originate a packet with
a given IP address and which ports could send a packet to a given IP
destination. Given that, you want the matching RDMA device.

Such a linkage would allow the application to correctly determine
the exact DAPL Provider that needed to be opened, and only only
that one. Without it the application has to scan the registry list
and essentially do a serial search. The good news is that it won't
be a very long serial search and it doesn't have to be performed
that often.