<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7638.1">
<TITLE>RE: [openib-general] openMPI for 2.6.17.10 kernel</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->
<P><FONT SIZE=2>Steve,<BR>
<BR>
As you know, I have my rnfs kernel running the stable iwarp-stack on my cluster now. But how do I compile the userspace packages from that stack? <BR>
<BR>
Thanks,<BR>
Helen<BR>
<BR>
-----Original Message-----<BR>
From: Steve Wise [<A HREF="mailto:swise@opengridcomputing.com">mailto:swise@opengridcomputing.com</A>]<BR>
Sent: Thu 11/30/2006 1:10 PM<BR>
To: Jeff Squyres<BR>
Cc: Chen, Helen Y; openib-general@openib.org; Leininger, Matthew L<BR>
Subject: Re: [openib-general] openMPI for 2.6.17.10 kernel<BR>
<BR>
On Thu, 2006-11-30 at 15:03 -0500, Jeff Squyres wrote:<BR>
> On Nov 30, 2006, at 2:58 PM, Steve Wise wrote:<BR>
><BR>
> >> // initiatior<BR>
> >> ibv_create_qp(...);<BR>
> >> ibv_modify_qp(...);// set pkey, port num, etc.<BR>
> >> send_via_oob_channel(lid, mtu, my_port_num, ...other info...);<BR>
> >> wait_for_oob_reply(...);<BR>
> >> ibv_modify_qp(...);//set mtu, lid, remote_port #, etc.<BR>
> >><BR>
> >> Are you telling me that the *ONLY* way to make QPs over iWARP<BR>
> >> connections is to use the RDMA CM, and it is not possible to work<BR>
> >> using a scheme like i listed above (passing relevant info via OOB<BR>
> >> channels)?<BR>
> ><BR>
> > Yup.<BR>
><BR>
> Then why did Tom Tucker tell me otherwise last week? Hrm. Tom -- <BR>
> did I not explain our current scheme well last week?<BR>
><BR>
<BR>
There must have been some mis-communication.<BR>
<BR>
> >> If so, I'm curious: what magic is the RDMA CM doing such<BR>
> >> that this is necessary?<BR>
> ><BR>
> > Setting up an TCP connection and negotiating RDMA mode.<BR>
><BR>
> Are you telling me that ibv_create_qp() and ibv_modify_qp() don't <BR>
> actually make a connection?<BR>
><BR>
<BR>
Yes. The QP is created with ibv_create_qp(). The TCP connection is<BR>
setup and bound to the QP via rdma_connect() or<BR>
rdma_listen()/rdma_accept().<BR>
<BR>
<BR>
Steve.<BR>
<BR>
<BR>
</FONT>
</P>
</BODY>
</HTML>