[ewg] Sharing Infiniband connection with forked child

[Richard Croucher]

Even with sockets, sharing the FD is frought with problems if more than
one process attempts to read the same socket.   There are ioctls to
allow you to peek but it gets pretty messy.  

As soon as you add RDMA into the mix it gets even more ugly.  What do
you do  to the memory regions you've registered?

It's much better to give all processes their own socket or  QP.  

If you do want to see the same data stream, then use multicast
addresses.   Each receiver then has their own, independent sockets/QP's.
It  does not  solve the RDMA issues since you are restricted to
unreliable datagrams but it does solve the sharing problems

You should also look at RDS.  This is a more elegant approach of sending
one2many.   As long as the receiving socket it there, it guarantees
delivery.

Richard


On Thu, 2011-01-13 at 16:00 -0800, Woodruff, Robert J wrote:

> No. If you call fork()/exec(), the new process would need
> To re-establish a new connection, not like sockets where
> an FD can be inherited.
> 
> -----Original Message-----
> From: Greg Kerr [mailto:kerr.g at husky.neu.edu] 
> Sent: Thursday, January 13, 2011 3:58 PM
> To: Woodruff, Robert J
> Cc: ewg at lists.openfabrics.org; Kapil Arya
> Subject: Re: [ewg] Sharing Infiniband connection with forked child
> 
> This is really helpful, thank you.
> 
> Out of curiosity, do you know, if I call exec() can the new process still 
> access the opened connection? Obviously any data structures setup (such as an 
> ibv_device or ibv_context) would be destroyed from memory, but perhaps I could 
> still access the connection somehow or maybe I serialized the old structures.
> 
> Thanks
> 
> On Thursday, January 13, 2011 06:45:16 pm Woodruff, Robert J wrote:
> > Greg wrote,
> > 
> > >I've been working with Infiniband recently and am relatively unfamiliar
> > >with the technology. I was hoping someone on this list could answer a
> > >question for me.
> > >
> > >When an Infiniband connection is established between a client and
> > >server, when  the client fork()s into a child process, can the child
> > >process share the existing connection with the parent process
> > >(as is the case with shared sockets)?
> > 
> > No. that is not allowed. The child can create new resources and
> > establish it's own communication, but it cannot inherit the parent's
> > connection.
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20110114/4ad42be6/attachment.html>