[Openib-windows] IBAT vs. ATS performance

[Tzachi Dar]

One more small issues that I have just thought of:
>From experiments that I have made, it seems that SendArp from multiple
threads to the same ip fails. I'm not sure if there is a real life issue
here, but a test that creates 2 threads and connects to some IP
sometimes fails on this (this also happens on Ethernet addresses).
Things are of course worse if there are 50 threads involved. As a result
some caching mechanism, or at least a mechanism that will prevent some
queries at the same time is needed. This increases the need for some
kind of a cache in any case.


>-----Original Message-----
>From: Tzachi Dar 
>Sent: Tuesday, December 13, 2005 6:54 PM
>To: Fab Tillier; openib-windows at openib.org
>Subject: RE: [Openib-windows] IBAT vs. ATS performance
>
>
>
>
>>-----Original Message-----
>>From: Fab Tillier [mailto:ftillier at silverstorm.com]
>>Sent: Monday, December 12, 2005 11:43 PM
>>To: Tzachi Dar; openib-windows at openib.org
>>Subject: RE: [Openib-windows] IBAT vs. ATS performance
>>
>>
>>> From: Tzachi Dar [mailto:tzachid at mellanox.co.il]
>>> Sent: Saturday, December 10, 2005 11:24 PM
>>> 
>>> The way to do lookups in the arp table is simply by calling
>>> GetIpNetTable().
>>> 
>>> This table has the information that you need. However, some
>>provessing
>>> is still needed to get the data out.
>>
>>Not only does the table need to be searched, it contains
>>entries for all adapters, and the adapter index is the only 
>>indication about which adapter entries correspond to.  This 
>>means that the adapter index needs to be correlated with the 
>>IPoIB instance.
>>
>>Once we have the interface index in user-mode, we don't need
>>to have IBAT provide the list of IP addresses anymore - we can 
>>just get those through the IP helper APIs too.  IBAT would 
>>then only need to provide MAC to GID translation.
>>
>>It really stinks that to make IBAT perform decently, it
>>requires a lot of extra code.
>>
>While we were still developing IBAT I noticed that there was a 
>way to do this using exsisting windows infrastructure.
>
>See: 
>http://openib.org/pipermail/openib-windows/2005-September/000219.html
>
>Still, I believe that doing it by ourselves is somewhat 
>simpler, and has a smaller chance of breaking in newer 
>versions of windows. You can try to find out if this approach 
>is faster or not.
>
>
>>> I'm not sure what the new performance will be but you will
>>not have to
>>> go to the network for this.
>>
>>GetIpNetTable (and all the associated post-processing) should
>>be faster, hopefully, than querying the SA.
>>
>>> It seems that the best solution will be to get the ARP table by
>>> calling
>>> GetIpNetTable() and creating a hash table based on the information
>>> there. The hash will promise optimal search time. From every time to
>>> time, you will have to refresh the table.
>>
>>I don't know if I want to go that far - caching the data
>>introduces lifetime and refresh issues that I'd rather stay 
>>away from.  The current code does ATS queries to the SA every 
>>time, so hopefully calling GetIpNetTable would be faster.
>>
>>I can't imagine that there isn't code to do something like
>>SendARP, but just hitting the ARP cache.  In any case, since 
>>using IBAT is going to be such a PITA, I think I'll put 
>>together an IBAT helper library in user-mode that wraps all that junk.
>>
>>Which of the IBAT IOCTLs does SDP issue from user-mode vs.
>>from kernel-mode? Would a user-mode IBAT library be useful to SDP?
>>
>Currently SDP is using IBAT only form kernel mode, as I 
>realized this will make things simpler for me (at least one of 
>the calls has to be done from kernel).
>
>Currently, I'm in a stage of implmenting basic functionality, 
>and I don't care of performance of things that are not the 
>data pass. Still, I'll probably be looking at this things in 
>the future. 
>
>My current implmentation calls SendArp from the user, and with 
>this result, it continues, giving the kernel module the needed 
>MAC (the kernel module than converts it to GID). In the 
>future, it might be nice to have a mechanism that does things 
>from the user.
>
>What functionality were you thinking to provide?
>
>
>>Thanks,
>>
>>- Fab
>>
>_______________________________________________
>openib-windows mailing list
>openib-windows at openib.org 
>http://openib.org/mailman/listinfo/openib->windows
>