[Openib-windows] IBAT vs. ATS performance

[Tzachi Dar]

>-----Original Message-----
>From: Fab Tillier [mailto:ftillier at silverstorm.com] 
>Sent: Monday, December 12, 2005 11:43 PM
>To: Tzachi Dar; openib-windows at openib.org
>Subject: RE: [Openib-windows] IBAT vs. ATS performance
>
>
>> From: Tzachi Dar [mailto:tzachid at mellanox.co.il]
>> Sent: Saturday, December 10, 2005 11:24 PM
>> 
>> The way to do lookups in the arp table is simply by calling 
>> GetIpNetTable().
>> 
>> This table has the information that you need. However, some 
>provessing 
>> is still needed to get the data out.
>
>Not only does the table need to be searched, it contains 
>entries for all adapters, and the adapter index is the only 
>indication about which adapter entries correspond to.  This 
>means that the adapter index needs to be correlated with the 
>IPoIB instance.
>
>Once we have the interface index in user-mode, we don't need 
>to have IBAT provide the list of IP addresses anymore - we can 
>just get those through the IP helper APIs too.  IBAT would 
>then only need to provide MAC to GID translation.
>
>It really stinks that to make IBAT perform decently, it 
>requires a lot of extra code.
>
While we were still developing IBAT I noticed that there was a way to do
this using exsisting windows infrastructure.

See:
http://openib.org/pipermail/openib-windows/2005-September/000219.html

Still, I believe that doing it by ourselves is somewhat simpler, and has
a smaller chance of breaking in newer versions of windows.
You can try to find out if this approach is faster or not.


>> I'm not sure what the new performance will be but you will 
>not have to 
>> go to the network for this.
>
>GetIpNetTable (and all the associated post-processing) should 
>be faster, hopefully, than querying the SA.
>
>> It seems that the best solution will be to get the ARP table by 
>> calling
>> GetIpNetTable() and creating a hash table based on the information
>> there. The hash will promise optimal search time. From every time to
>> time, you will have to refresh the table.
>
>I don't know if I want to go that far - caching the data 
>introduces lifetime and refresh issues that I'd rather stay 
>away from.  The current code does ATS queries to the SA every 
>time, so hopefully calling GetIpNetTable would be faster.
>
>I can't imagine that there isn't code to do something like 
>SendARP, but just hitting the ARP cache.  In any case, since 
>using IBAT is going to be such a PITA, I think I'll put 
>together an IBAT helper library in user-mode that wraps all that junk.
>
>Which of the IBAT IOCTLs does SDP issue from user-mode vs. 
>from kernel-mode? Would a user-mode IBAT library be useful to SDP?
>
Currently SDP is using IBAT only form kernel mode, as I realized this
will make things simpler for me (at least one of the calls has to be
done from kernel).

Currently, I'm in a stage of implmenting basic functionality, and I
don't care of performance of things that are not the data pass. Still,
I'll probably be looking at this things in the future. 

My current implmentation calls SendArp from the user, and with this
result, it continues, giving the kernel module the needed MAC (the
kernel module than converts it to GID). In the future, it might be nice
to have a mechanism that does things from the user.

What functionality were you thinking to provide?


>Thanks,
>
>- Fab
>