[Openib-windows] IBAT vs. ATS performance

[Fab Tillier]

> From: Tzachi Dar [mailto:tzachid at mellanox.co.il]
> Sent: Saturday, December 10, 2005 11:24 PM
> 
> The way to do lookups in the arp table is simply by calling
> GetIpNetTable().
> 
> This table has the information that you need. However, some provessing
> is still needed to get the data out.

Not only does the table need to be searched, it contains entries for all
adapters, and the adapter index is the only indication about which adapter
entries correspond to.  This means that the adapter index needs to be correlated
with the IPoIB instance.

Once we have the interface index in user-mode, we don't need to have IBAT
provide the list of IP addresses anymore - we can just get those through the IP
helper APIs too.  IBAT would then only need to provide MAC to GID translation.

It really stinks that to make IBAT perform decently, it requires a lot of extra
code.

> I'm not sure what the new performance will be but you will not have to
> go to the network for this.

GetIpNetTable (and all the associated post-processing) should be faster,
hopefully, than querying the SA.

> It seems that the best solution will be to get the ARP table by calling
> GetIpNetTable() and creating a hash table based on the information
> there. The hash will promise optimal search time. From every time to
> time, you will have to refresh the table.

I don't know if I want to go that far - caching the data introduces lifetime and
refresh issues that I'd rather stay away from.  The current code does ATS
queries to the SA every time, so hopefully calling GetIpNetTable would be
faster.

I can't imagine that there isn't code to do something like SendARP, but just
hitting the ARP cache.  In any case, since using IBAT is going to be such a
PITA, I think I'll put together an IBAT helper library in user-mode that wraps
all that junk.

Which of the IBAT IOCTLs does SDP issue from user-mode vs. from kernel-mode?
Would a user-mode IBAT library be useful to SDP?

Thanks,

- Fab