[ofa-general] Re: Sharing userspace IB objects

[Michael S. Tsirkin]

> Quoting Roland Dreier <rdreier at cisco.com>:
> Subject: Re: Sharing userspace IB objects
> 
>  > Can you please elaborate a little bit more on what steps are required to
>  > achieve this? I have a connection manager running as a separate process from
>  > the apps which would be sending/receiving data on QPs. I was hoping to
>  > create IB objects via CM and be made sharable to the apps.
> 
> You would have to do a lot of hacking of low-level stuff (libibverbs
> and whatever userspace driver libraries you need) to handle passing
> file descriptors through unix domain sockets and figure out a way to
> make the CQ/QP buffers visible in the address space of the process
> that will actually use them.  And also handle doorbell pages etc.

This is related to scalability stuff that Dror presented at Sonoma
http://www.openfabrics.org/archives/spring2007sonoma/Tuesday%20May%201/gdror%20Next%20Generation%20Hardware%20Assists%20And%20Scalability2.pdf

See especially the shared send queue slide.

So, since the need seems to be there, I started thinking about how this could be done.
Basically, we could create shared memory objects (shm_open) and use these
for all hardware-accessible registers, as well as necessary control (head/tail pointers,
spinlocks used for protection, etc).

If we do this, we can use unix domain sockets for everything,
a client just mmaps the fd that it got. Does this make sense?

-- 
MST