[openfabrics-ewg] rc3 installation issues.

Mon May 1 11:29:01 PDT 2006

Scott,

Thanks for testing Intel MPI on OFED 1.0 rc3. Yes, the socket CM uDAPL
provider will work fine without uCMA and IPoIB installed. Any idea why
your latency is so high? It should be around 3-4us.

-arlin

________________________________

From: Scott Weitzenkamp (sweitzen) [mailto:sweitzen at cisco.com] 
Sent: Sunday, April 30, 2006 10:32 PM
To: Woodruff, Robert J; Aviram Gutman; Tziporet Koren; Amit Mehrotra
(amehrotr); openfabrics-ewg at openib.org
Cc: Alexander Smirnov; Davis, Arlin R
Subject: RE: [openfabrics-ewg] rc3 installation issues.

Intel MPI 2.0 does work with OFED 1.0 rc3 on RHEL4 U3 despite the lack
of uCMA, here's example output on x86_64 w/PCI-E:

.../osu_latency.x
CMA: unable to open /dev/infiniband/rdma_cm
CMA: unable to open /dev/infiniband/rdma_cm
CMA: unable to open /dev/infiniband/rdma_cm
CMA: unable to open /dev/infiniband/rdma_cm
I_MPI: [0] set_up_devices(): will use device: libmpi.rdma.so
I_MPI: [0] set_up_devices(): will use DAPL provider: OpenIB-scm1
CMA: unable to open /dev/infiniband/rdma_cm
CMA: unable to open /dev/infiniband/rdma_cm
CMA: unable to open /dev/infiniband/rdma_cm
CMA: unable to open /dev/infiniband/rdma_cm
I_MPI: [0] set_up_devices(): will use device: libmpi.rdma.so
I_MPI: [0] set_up_devices(): will use DAPL provider: OpenIB-scm1
# OSU MPI Latency Test (Version 2.1)
# Size          Latency (us) 
0               7.86
1               7.53
2               7.37
4               7.43
8               7.92
16              7.29
32              7.36
64              7.50

Scott Weitzenkamp

SQA and Release Manager

Server Virtualization Business Unit

Cisco Systems

________________________________

	From: openfabrics-ewg-bounces at openib.org
[mailto:openfabrics-ewg-bounces at openib.org] On Behalf Of Woodruff,
Robert J
	Sent: Tuesday, April 25, 2006 9:16 AM
	To: Aviram Gutman; Tziporet Koren; Amit Mehrotra (amehrotr);
openfabrics-ewg at openib.org
	Cc: Alexander Smirnov; Davis, Arlin R
	Subject: RE: [openfabrics-ewg] rc3 installation issues.

	Intel MPI needs a stable uDAPL and supporting usermode
verbs/uCMA and such.

	If you have that, then Intel MPI should work. If you want to get
a copy of Intel MPI

	to use in your testing of OFED, I can put you in contact with
our Intel MPI lead and he can

	provide you with the code and a license. 

________________________________

	From: Aviram Gutman [mailto:aviram at mellanox.co.il] 
	Sent: Tuesday, April 25, 2006 9:09 AM
	To: Woodruff, Robert J; Tziporet Koren; Amit Mehrotra
(amehrotr); openfabrics-ewg at openib.org
	Cc: Alexander Smirnov; Davis, Arlin R
	Subject: RE: [openfabrics-ewg] rc3 installation issues.

	Please work with Vlad to make sure you have what you need in
rc4.

	Regards,

	   Aviram

________________________________

	From: Woodruff, Robert J [mailto:robert.j.woodruff at intel.com] 
	Sent: Tuesday, April 25, 2006 6:56 PM
	To: Aviram Gutman; Tziporet Koren; Amit Mehrotra (amehrotr);
openfabrics-ewg at openib.org
	Cc: Alexander Smirnov; Davis, Arlin R
	Subject: RE: [openfabrics-ewg] rc3 installation issues.

	Not yet. Last I heard, there was no uDAPL in OFED that is neeed
by Intel MPI. Maybe RC4. 

	 Tziporet wrote,

	> Regarding uDAPL and RHEL4 (up2 & 3): Since we have not
completed the uCMA backport for kernel 2.6.9 we eliminated uDAPL  

________________________________

	From: Aviram Gutman [mailto:aviram at mellanox.co.il] 
	Sent: Tuesday, April 25, 2006 8:48 AM
	To: Woodruff, Robert J; Tziporet Koren; Amit Mehrotra
(amehrotr); openfabrics-ewg at openib.org
	Cc: Alexander Smirnov; Davis, Arlin R
	Subject: RE: [openfabrics-ewg] rc3 installation issues.

	Woody,

	Are you testing Intel MPI on top of OFED?

	Regards,

	   Aviram

________________________________

	From: openfabrics-ewg-bounces at openib.org
[mailto:openfabrics-ewg-bounces at openib.org] On Behalf Of Woodruff,
Robert J
	Sent: Monday, April 24, 2006 11:33 PM
	To: Tziporet Koren; Amit Mehrotra (amehrotr);
openfabrics-ewg at openib.org
	Cc: Alexander Smirnov; Davis, Arlin R
	Subject: RE: [openfabrics-ewg] rc3 installation issues.

	The latest patches I have is for SVN 6509 on the trunk. 

	https://openib.org/svn/gen2/branches/backport-to-2.6.9/

	infiniband-backport-svn6509-to-2.6.9-34.EL-kernel-fixups-00.diff
	infiniband-backport-svn6509-to-2.6.9-34.EL-kernel-fixups-01.diff
	infiniband-backport-svn6509-to-2.6.9-openib-drivers-02.diff
	infiniband-backport-svn6509-to-2.6.9-openib-fixups-03.diff

	I don't split the patches by component, but rather have one to
strip out the old IB

	code in RedHat EL4.0, one to apply needed kernel changes to the
RedHat kernel,

	one for the openib drivers, and one for openib fixups that are
needed.

________________________________

	From: Tziporet Koren [mailto:tziporet at mellanox.co.il] 
	Sent: Monday, April 24, 2006 11:43 AM
	To: Woodruff, Robert J; Amit Mehrotra (amehrotr);
openfabrics-ewg at openib.org
	Cc: Alexander Smirnov; Davis, Arlin R
	Subject: RE: [openfabrics-ewg] rc3 installation issues.

	We plan to do a backport of uCMA for RC4.

	Can you point me to your backport of uCMA for 2.6.9-34EL

	Thanks,

	Tziporet

	-----Original Message-----
	From: Bob Woodruff [mailto:robert.j.woodruff at intel.com] 
	Sent: Monday, April 24, 2006 6:56 PM
	To: Tziporet Koren; Amit Mehrotra (amehrotr);
openfabrics-ewg at openib.org
	Cc: Alexander Smirnov; Davis, Arlin R
	Subject: RE: [openfabrics-ewg] rc3 installation issues.

	 Tziporet wrote,

	> Regarding uDAPL and RHEL4 (up2 & 3): Since we have not
completed the uCMA backport for kernel 2.6.9 we eliminated uDAPL  

	>  installation in case of this kernel. We also indicated this
in the package limitation mail. 

	Note that there are two uDAPL providers for OpenIB. One uses
standard sockets for connections (socket CM)

	and the other uses CMA. I would suggest that you should include
the socket CM uDAPL provider even

	if you do not have a backport for uCMA.

	BTW. - My backport patches contain backport for uCMA for
2.6.9-34-EL if you need an example.

	woody

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20060501/90307f06/attachment.html>