[openib-general] in need of a simple ulp

[Jeff Carr]

I'm new to IB so over the last few weeks I've read a large part of the 
openib archives. I have a small IB cluster of a few nodes running under 
2.6.11.7. IPoIB is working ok (SM dedicated on another node running 
2.6.9 & Mellanox IBGold).

To solve my particular problem, it would seem to me the best option 
would be to write a small ulp for this purpose. In this case, I have 
some pages in memory on one host that I need to offload to the other IB 
hosts for more cpu intensive processing. So, this is very similar to the 
way perf_main/simple_perf works, but I need a kernel space implementation.

IBoIP is rather performance bound it would seem. I was hoping that I 
could write/find some simple code to help with this, but there are not 
many ulp's and I notice from a previous thread last month there doesn't 
seem to be much abstraction or shared code that can be used.

In my infant state of understanding IB, it would appear that I need to

1) to open & discover the HCA's
2) create some Priority Domains
3) create some address handles
4) create some CQ's
5) create some queue pairs (maybe some SRQ's also)

possibly 3-5 can be replaced with some sort of "Registered Memory 
Region" as per 11.2.8 of the IB spec. (?)

I've attempted to look in the subversion repository for anything close 
or for things to use as starting points but didn't seem to find anything 
yet. The full repository is large and not very cleanly organized for a 
new user; so if there is some starting code that may be of use, that is 
what I was looking for.

Also, if there is anyone that would be willing to work on this problem 
on a contractual basis (code will be GPL'd) then please contact me.

Thanks,
Jeff Carr


Some choice quotes from the archives:

Ah a "design spec".  I remember writing those.  And I remember
discussing them with other engineers for hours about the proper wording
for paticular sections, and other trivial nonsense.

... if you are talking to the right people what does the name of
the room you happen to be in happen to do with anything?

Somebody needs to work on a history someday, as a sort of "don't do 
this" tutorial for the industry :-)

I've done a NIC design or two and once made the same mistake myself of
thinking I could guarantee data integrity -- turned out I could not. It
was good to know I was in such good company.

You laugh when you mention Cosmic Rays. Well, don't.

I don't think there's anything "illegal" in developing an SDP 
implementation, either under the GPL or otherwise. However, shipping it 
in products is another thing entirely.