[openib-general] Looking for a Mellanox VAPI example C/C++ code

[Sinate]

Thanks Mike. I'll put your views to my boss who is asking me to write 
the VAPI app. So far I've also used SDP, with some pretty reasonable 
results. However, the customer always wants more. I've also put this 
question to Infinicon. I won't be surprised if I don't get a response.

Guess I better start searching OpenIB archive...

Cheers,

Sinate.

Mike Houston wrote:

> This has been a running request now for quite some time.  Everyone 
> says its a good idea, but it's hard to get it actually done.  (And 
> before people say, "this is an open source project so why don't you do 
> it yourself", keep reading)  Since the spec is slowly evolving, we 
> really need the OpenIB developers or the IBTA to write a programmer's 
> manual for VAPI.  Folks working on the ULPs should do the same.
> I learned (poorly) how to use VAPI by reading the original Mellanox 
> perf_main code, which was a truly awful experience.  This has lead to 
> me writing a VERY fragile NAL using VAPI that has performance 
> characteristics which are very difficult to reason about.
> Without updated API documentation and programming guides, IB is going 
> to be a hard sell to many folks unless their important apps already 
> work perfectly an fast on IB somehow.  For example, what fields do I 
> set in the many different structs to get good performance?  The event 
> handlers are cool, but how do you actually use them?  How do I figure 
> out if a cool feature is actually supported, i.e. caps bits?  We need 
> something like a "HelloIB" for all of the different ways to access the 
> IB hardware (*DAPL, VAPI, SDP (with all the socket extensions, etc)), 
> along with a programmer's guide.  Something like if you had to teach 
> an intro undergrad course on networking, but instead of using TCP, you 
> had to teach using IB.  What would your slides look like?  What would 
> the homework/project questions and solutions be...  i.e UD vs. RC.  
> send/recv vs. RDMA (read/write).  Basic explanations of 
> WQEs/SRQs/CQs/QPs/etc.
>
> If you look at the OpenIB archive of this list, you can find an 
> example or two of how to write a ping test.  But, getting good 
> performance through VAPI seems to still be a dark art.  It's obviously 
> doable since the OSU folks are getting pretty good performance with 
> their MVAPICH MPI implementation on IB.  I can get pretty good 
> performance point to point, but 1->N, N->1, M->N, and all to all 
> continue to be a problem for me.
>
> This is why I've made heavy use out of SDP, but to get all the 
> performance of IB, especially on PCIe, it looks like we will have to 
> write code pretty close to the metal, be it VAPI or kDAPL(?).
>
> This is just my opinion as and end user, I could be wrong.
>
> -Mike
>
> Sinate wrote:
>
>> Hi. I'm trying to find a simple Mellanox VAPI example (sends some 
>> data from one host to another, nothing fancy), in C or C++.
>>
>> I can stare at the Mellanox VAPI reference files, but it doesn't get 
>> me any where if I want to build a C/C++ program to do something.
>>
>> Sorry if this is the wrong place to ask.
>>
>> Cheers.
>> _______________________________________________
>> openib-general mailing list
>> openib-general at openib.org
>> http://openib.org/mailman/listinfo/openib-general
>>
>> To unsubscribe, please visit 
>> http://openib.org/mailman/listinfo/openib-general
>
>
>