[openib-general] openMPI for 2.6.17.10 kernel

[Tom Tucker]

On Thu, 2006-11-30 at 15:03 -0500, Jeff Squyres wrote:
> On Nov 30, 2006, at 2:58 PM, Steve Wise wrote:
> 
> >> 	// initiatior
> >> 	ibv_create_qp(...);
> >> 	ibv_modify_qp(...);// set pkey, port num, etc.
> >> 	send_via_oob_channel(lid, mtu, my_port_num, ...other info...);
> >> 	wait_for_oob_reply(...);
> >> 	ibv_modify_qp(...);//set mtu, lid, remote_port #, etc.
> >>
> >> Are you telling me that the *ONLY* way to make QPs over iWARP
> >> connections is to use the RDMA CM, and it is not possible to work
> >> using a scheme like i listed above (passing relevant info via OOB
> >> channels)?
> >
> > Yup.
> 
> Then why did Tom Tucker tell me otherwise last week?  Hrm.  Tom --  
> did I not explain our current scheme well last week?

I certainly know we talked and I know you explained your current
out-of-band connection management scheme. I've got a reasonable
understanding of how the iWARP/TCP connection management stuff works, so
I'm pretty sure I didn't tell you that this kind of scheme would work
for iWARP. Sorry for the misunderstanding. 

> 
> >> If so, I'm curious: what magic is the RDMA CM doing such
> >> that this is necessary?
> >
> > Setting up an TCP connection and negotiating RDMA mode.
> 
> Are you telling me that ibv_create_qp() and ibv_modify_qp() don't  
> actually make a connection?
> 
> Don't get me wrong -- we're moving towards the RDMA CM for OMPI, but  
> I just want to understand what the problem is today, especially since  
> I got seemingly opposite information last week.
>