[openib-general] ibv_get_device_list() question

[Roland Dreier]

 > 1. Does the ibv API provide any information about devices' physical  
 > location with regards to busses / processors?  (I doubt it, but I'm  
 > just asking to be sure)

Not directly through the API, and I would hate to try and design an
API to encapsulate all that info.  I think chasing down stuff through
sysfs as was already suggested is your best bet.  If the sysfs files
exist you're happy and can try to optimize automatically, otherwise
you have to rely on what the user feeds you manually.

 > 2. Even if we get this information externally (e.g., user provides a  
 > config file to OMPI), does the information returned via  
 > ibv_get_device_list() come back in a guaranteed order?  I.e., if I  
 > have multiple HCAs in a host, will the entries in the list always be  
 > returned in the same order such that I could have a host-specific  
 > OMPI config file saying "processes on cores 0,1,2,3 use ibv_device 0,  
 > processes on cores 4,5,6,7 use device 1", or something similar?

No, in the presence of hotplug, differences in PCI bus scanning
between different kernels, all of that can change.  I think the sane
way to refer to IB devices is with GUIDs, since those are supposed to
be pretty persistent.

 - R.