<div>Sundeep,</div> <div> </div> <div>I do not see any subdirectory for lib64 or lib in either the iWARP branch or in the OFED branch.</div> <div>Can you please tell me the reference for the OPEN_IB_LIB in the iWARP MVAPICH2 makefile?</div> <div> </div> <div>Regards,</div> <div>David</div> <div> </div> <div><BR><BR><B><I>david elsen <elsen_david@yahoo.com></I></B> wrote:</div> <BLOCKQUOTE class=replbq style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px solid"> <DIV>The iWARP code does not have these directories. </DIV> <DIV> </DIV> <DIV>Will it be better if the iWARP makefile is not dependent on the OFED code?<BR><BR><B><I>david elsen <elsen_david@yahoo.com></I></B> wrote:</DIV> <BLOCKQUOTE class=replbq style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px solid"> <DIV>I modified the OPEN_IB_HOME to the iWARP directroy. Now it is looking </DIV> <DIV>OPEN_IB_HOME/lib64 and OPEN_IB_HOME/lib
directories.<BR><BR><B><I>Sundeep Narravula <narravul@cse.ohio-state.edu></I></B> wrote:</DIV> <BLOCKQUOTE class=replbq style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px solid">Hi David,<BR><BR>> My question is why should I have this reference to /usr/local/ofed<BR>> there if I do not need to download the OFED distribution code to run<BR>> the iWARP.<BR><BR>The variable OPEN_IB_HOME in make.mvapich2.iwarp sets the path to the Gen2<BR>installation that you intend to use for iwarp. The default in the script<BR>is /usr/local/ofed.<BR><BR>Based on your installation, please set this variable appropriately.<BR><BR>export OPEN_IB_HOME=/usr/local<BR><BR>> Is it possible to add more information in your MVAPICH2 0.9.8 User Guide describing how to build this and what are the dependencies?<BR><BR>We have updated our userguide with the installation information. The<BR>dependencies that we have are the installation of OF iwarp branch, the<BR>setup
of rdma-cm module and the setup of the underlying network.<BR><BR>Regards,<BR>--Sundeep.<BR><BR>><BR>> David<BR>> Sundeep Narravula <NARRAVUL@CSE.OHIO-STATE.EDU>wrote:<BR>> Hi David,<BR>><BR>> > iWARP is actually a part of the Open Fabrics SVN. It is available from<BR>> > a different branch.<BR>> ><BR>> > I am cc'ing this note to my group. One of the students (Sundeep) will<BR>> > send you the detailed instructions on which branch of OF to download<BR>> > and use.<BR>><BR>> The instructions for setting up iwarp on OpenFrabics is avalable on the<BR>> openIB wiki at<BR>> https://openib.org/tiki/tiki-index.php?page=Install+OpenIB+for+Chelsio+T3<BR>> https://openib.org/tiki/tiki-index.php?page=Install+OpenIB+for+Ammasso1100<BR>><BR>> Further, the branch you can download from the svn is<BR>> https://openib.org/svn/gen2/branches/iwarp/tags/iwarp-2.6.17-stable<BR>><BR>> Please let us know if you have
any further questions.<BR>><BR>> Regards,<BR>> --Sundeep.<BR>><BR>><BR>> > > Is there any document describing the build process of the MVAPICH2<BR>> > > tool? I am going through the MVAPICH2 0.9.8 users guide and that<BR>> > > does not seem to be giving me the detailed information.<BR>> ><BR>> > We will add this additional information on our user guide.<BR>> ><BR>> > > Can you please provide README files for the iWARP which describes<BR>> > > TODO steps?<BR>> ><BR>> > Sundeep's information will help. If you have any additional questions,<BR>> > please feel free to ask us.<BR>> ><BR>> > Thanks,<BR>> ><BR>> > DK<BR>> ><BR>> > > Thanks,<BR>> > > David<BR>> > ><BR>> > > Dhabaleswar Panda<BR>> wrote:<BR>> > > The MVAPICH team is pleased to announce the availability of MVAPICH2<BR>> > > 0.9.8 with
the following NEW features:<BR>> > ><BR>> > > - Checkpoint/Restart support for application transparent systems-level<BR>> > > fault tolerance. BLCR-based support using native InfiniBand Gen2<BR>> > > interface is provided. Flexible interface to work with different<BR>> > > file systems. Tested with ext3 (local disk), NFS and PVFS2.<BR>> > ><BR>> > > Performance of sample applications with checkpoint-restart using<BR>> > > PVFS2 and Lustre can be found here:<BR>> > ><BR>> > > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/performance/mvapich2/application/MVAPICH2-ckpt.html<BR>> > ><BR>> > > - iWARP support: Incorporates the support for OpenFabrics/Gen2-iWARP.<BR>> > > Tested with Chelsio T3 (10GigE) and Ammasso iWARP adapters and<BR>> > > drivers.<BR>> > ><BR>> > > - RDMA CM-based Connection management support<BR>> >
><BR>> > > - Shared memory optimizations for collective communication operations.<BR>> > > Efficient algorithms and optimizations for barrier, reduce and<BR>> > > all-reduce operations. Exploits the multi-core optimized shared<BR>> > > memory point-to-point communication support introduced in MVAPICH2<BR>> > > 0.9.6.<BR>> > ><BR>> > > Performance of sample collective operations with this new feature<BR>> > > can be found here:<BR>> > ><BR>> > > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/perf-coll.html<BR>> > ><BR>> > > - uDAPL support for NetEffect 10GigE adapter. Tested with<BR>> > > NetEffect NE010 adapter.<BR>> > ><BR>> > > More details on all features and supported platforms can be obtained<BR>> > > by visiting the following URL:<BR>> > ><BR>> > >
http://nowlab.cse.ohio-state.edu/projects/mpi-iba/mvapich2_features.html<BR>> > ><BR>> > > MVAPICH2 0.9.8 release is tested with the latest OFED 1.1 stack. It<BR>> > > continues to deliver excellent performance. Sample performance<BR>> > > numbers include:<BR>> > ><BR>> > > - OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR:<BR>> > > Two-sided operations:<BR>> > > - 2.81 microsec one-way latency (4 bytes)<BR>> > > - 1561 MB/sec unidirectional bandwidth<BR>> > > - 2935 MB/sec bidirectional bandwidth<BR>> > ><BR>> > > One-sided operations:<BR>> > > - 4.92 microsec Put latency<BR>> > > - 1569 MB/sec unidirectional Put bandwidth<BR>> > > - 2935 MB/sec bidirectional Put bandwidth<BR>> > ><BR>> > > - OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR (Dual-rail):<BR>> > > Two-sided
operations:<BR>> > > - 2.81 microsec one-way latency (4 bytes)<BR>> > > - 3127 MB/sec unidirectional bandwidth<BR>> > > - 5917 MB/sec bidirectional bandwidth<BR>> > ><BR>> > > One-sided operations:<BR>> > > - 4.37 microsec Put latency<BR>> > > - 3137 MB/sec unidirectional Put bandwidth<BR>> > > - 5917 MB/sec bidirectional Put bandwidth<BR>> > ><BR>> > > - OpenFabrics/Gen2 on Opteron single-core with PCI-Ex and IBA-DDR:<BR>> > > Two-sided operations:<BR>> > > - 3.01 microsec one-way latency (4 bytes)<BR>> > > - 1402 MB/sec unidirectional bandwidth<BR>> > > - 2238 MB/sec bidirectional bandwidth<BR>> > ><BR>> > > One-sided operations:<BR>> > > - 4.65 microsec Put latency<BR>> > > - 1402 MB/sec unidirectional Put bandwidth<BR>> > > - 2238 MB/sec bidirectional Put bandwidth<BR>> > ><BR>> > >
Performance numbers for all other platforms, system configurations and<BR>> > > operations can be viewed by visiting `Performance' section of the<BR>> > > project's web page.<BR>> > ><BR>> > > With the ADI-3-level design, MVAPICH2 0.9.8 delivers similar<BR>> > > performance for two-sided operations compared to MVAPICH 0.9.8.<BR>> > > Organizations and users interested in getting the best performance for<BR>> > > both two-sided and one-sided operations and also want to exploit<BR>> > > advanced features (such as fault tolerance with checkpoint/restart,<BR>> > > iWARP, RDMA CM connection management, multi-threading, integrated<BR>> > > multi-rail, multi-core optimization, memory hook support and optimized<BR>> > > collectives) may migrate from MVAPICH code base to MVAPICH2 code base.<BR>> > ><BR>> > > For downloading MVAPICH2 0.9.8 package and accessing the
anonymous<BR>> > > SVN, please visit the following URL:<BR>> > ><BR>> > > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/<BR>> > ><BR>> > > A stripped down version of this release is also available at the<BR>> > > OpenFabrics SVN.<BR>> > ><BR>> > > All feedbacks, including bug reports and hints for performance tuning,<BR>> > > are welcome. Please post it to the mvapich-discuss mailing list.<BR>> > ><BR>> > > Thanks,<BR>> > ><BR>> > > MVAPICH Team at OSU/NBCL<BR>> > ><BR>> > > ======================================================================<BR>> > > MVAPICH/MVAPICH2 project is currently supported with funding from<BR>> > > U.S. National Science Foundation, U.S. DOE Office of Science,<BR>> > > Mellanox, Intel, Cisco Systems, Sun Microsystems and Linux Networx;<BR>> > > and with equipment support
from Advanced Clustering, AMD, Apple,<BR>> > > Appro, Dell, IBM, Intel, Mellanox, Microway, PathScale, SilverStorm<BR>> > > and Sun Microsystems. Other technology partner includes Etnus.<BR>> > > ======================================================================<BR>> > ><BR>> > > _______________________________________________<BR>> > > openfabrics-ewg mailing list<BR>> > > openfabrics-ewg@openib.org<BR>> > > http://openib.org/mailman/listinfo/openfabrics-ewg<BR>> > ><BR>> > ><BR>> > ><BR>> > ><BR>> > > ---------------------------------<BR>> > > Sponsored Link<BR>> > ><BR>> > > Try Netflix today! With plans starting at only $5.99 a month what are you waiting for?<BR>> > > --0-119921422-1163376356=:19890<BR>> > > Content-Type: text/html; charset=iso-8859-1<BR>> > > Content-Transfer-Encoding:
8bit<BR>> > ><BR>> > > I am trying to use the OSC MPI tool for the iWARP and quite new to the open fabrics tools.<BR>><BR>> If I know it correctly, iWARP is not yet part of the OFED release.<BR>> But the iWARP makefile has reference to ofed code. Is it really required.<BR>><BR>> Is there any document describing the build process of the MVAPICH2 tool? I am going through the MVAPICH2 0.9.8 users guide and that does not seem to be giving me the detailed information.<BR>><BR>> Can you please provide README files for the iWARP which describes TODO steps?<BR>><BR>> Thanks,<BR>> David<BR>><BR>> Dhabaleswar Panda <PANDA@CSE.OHIO-STATE.EDU>wrote:<BR>> The MVAPICH team is pleased to announce the availability of MVAPICH2<BR>> 0.9.8 with the following NEW features:<BR>><BR>> -<BR>> > > Checkpoint/Restart support for application transparent systems-level<BR>> fault tolerance. BLCR-based support using native
InfiniBand Gen2<BR>> interface is provided. Flexible interface to work with different<BR>> file systems. Tested with ext3 (local disk), NFS and PVFS2.<BR>><BR>> Performance of sample applications with checkpoint-restart using<BR>> PVFS2 and Lustre can be found here:<BR>><BR>> http://nowlab.cse.ohio-state.edu/projects/mpi-iba/performance/mvapich2/application/MVAPICH2-ckpt.html<BR>><BR>> - iWARP support: Incorporates the support for OpenFabrics/Gen2-iWARP.<BR>> Tested with Chelsio T3 (10GigE) and Ammasso iWARP adapters and<BR>> drivers.<BR>><BR>> - RDMA CM-based Connection management support<BR>><BR>> - Shared memory optimizations for collective communication operations.<BR>> Efficient algorithms and optimizations for barrier, reduce and<BR>> all-reduce operations. Exploits the multi-core optimized shared<BR>> memory point-to-point communication support introduced in<BR>> > > MVAPICH2<BR>>
0.9.6.<BR>><BR>> Performance of sample collective operations with this new feature<BR>> can be found here:<BR>><BR>> http://nowlab.cse.ohio-state.edu/projects/mpi-iba/perf-coll.html<BR>><BR>> - uDAPL support for NetEffect 10GigE adapter. Tested with<BR>> NetEffect NE010 adapter.<BR>><BR>> More details on all features and supported platforms can be obtained<BR>> by visiting the following URL:<BR>><BR>> http://nowlab.cse.ohio-state.edu/projects/mpi-iba/mvapich2_features.html<BR>><BR>> MVAPICH2 0.9.8 release is tested with the latest OFED 1.1 stack. It<BR>> continues to deliver excellent performance. Sample performance<BR>> numbers include:<BR>><BR>> - OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR:<BR>> Two-sided operations:<BR>> - 2.81 microsec one-way latency (4 bytes)<BR>> - 1561 MB/sec unidirectional bandwidth<BR>> - 2935 MB/sec bidirectional bandwidth<BR>><BR>> One-sided
operations:<BR>> - 4.92 microsec Put latency<BR>> - 1569 MB/sec unidirectional Put bandwidth<BR>> - 2935 MB/sec<BR>> > > bidirectional Put bandwidth<BR>><BR>> - OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR (Dual-rail):<BR>> Two-sided operations:<BR>> - 2.81 microsec one-way latency (4 bytes)<BR>> - 3127 MB/sec unidirectional bandwidth<BR>> - 5917 MB/sec bidirectional bandwidth<BR>><BR>> One-sided operations:<BR>> - 4.37 microsec Put latency<BR>> - 3137 MB/sec unidirectional Put bandwidth<BR>> - 5917 MB/sec bidirectional Put bandwidth<BR>><BR>> - OpenFabrics/Gen2 on Opteron single-core with PCI-Ex and IBA-DDR:<BR>> Two-sided operations:<BR>> - 3.01 microsec one-way latency (4 bytes)<BR>> - 1402 MB/sec unidirectional bandwidth<BR>> - 2238 MB/sec bidirectional bandwidth<BR>><BR>> One-sided operations:<BR>> - 4.65 microsec Put latency<BR>> - 1402 MB/sec unidirectional Put
bandwidth<BR>> - 2238 MB/sec bidirectional Put bandwidth<BR>><BR>> Performance numbers for all other platforms, system configurations and<BR>> operations can be viewed by visiting `Performance' section of the<BR>> project's web page.<BR>><BR>> With the<BR>> > > ADI-3-level design, MVAPICH2 0.9.8 delivers similar<BR>> performance for two-sided operations compared to MVAPICH 0.9.8.<BR>> Organizations and users interested in getting the best performance for<BR>> both two-sided and one-sided operations and also want to exploit<BR>> advanced features (such as fault tolerance with checkpoint/restart,<BR>> iWARP, RDMA CM connection management, multi-threading, integrated<BR>> multi-rail, multi-core optimization, memory hook support and optimized<BR>> collectives) may migrate from MVAPICH code base to MVAPICH2 code base.<BR>><BR>> For downloading MVAPICH2 0.9.8 package and accessing the anonymous<BR>> SVN, please visit
the following URL:<BR>><BR>> http://nowlab.cse.ohio-state.edu/projects/mpi-iba/<BR>><BR>> A stripped down version of this release is also available at the<BR>> OpenFabrics SVN.<BR>><BR>> All feedbacks, including bug reports and hints for performance tuning,<BR>> are welcome. Please post it to the mvapich-discuss mailing list.<BR>><BR>> Thanks,<BR>> > ><BR>><BR>> MVAPICH Team at OSU/NBCL<BR>><BR>> ======================================================================<BR>> MVAPICH/MVAPICH2 project is currently supported with funding from<BR>> U.S. National Science Foundation, U.S. DOE Office of Science,<BR>> Mellanox, Intel, Cisco Systems, Sun Microsystems and Linux Networx;<BR>> and with equipment support from Advanced Clustering, AMD, Apple,<BR>> Appro, Dell, IBM, Intel, Mellanox, Microway, PathScale, SilverStorm<BR>> and Sun Microsystems. Other technology partner includes Etnus.<BR>>
======================================================================<BR>><BR>> _______________________________________________<BR>> openfabrics-ewg mailing list<BR>> openfabrics-ewg@openib.org<BR>> http://openib.org/mailman/listinfo/openfabrics-ewg<BR>><BR>><BR>><BR>> > ><BR>> > ><BR>> > ><BR>> ---------------------------------<BR>> Sponsored Link<BR>><BR>><BR>> > > Try Netflix today! With plans starting at only $5.99 a month what are you waiting for?<BR>> > > --0-119921422-1163376356=:19890--<BR>> > ><BR>> ><BR>><BR>><BR>><BR>><BR>> ---------------------------------<BR>> Sponsored Link<BR>><BR>> $200,000 mortgage for $660/mo - 30/15 yr fixed, reduce debt, home equity - Click now for info<BR><BR><BR></BLOCKQUOTE><BR> <DIV> <HR SIZE=1> Sponsored Link<BR><BR><A
href="http://o1.qnsr.com/cgi/r?;n=203;c=232538;s=2014;x=7936;f=200611011404190;u=j;z=TIMESTAMP;">Get an Online or Campus degree - Associate's, Bachelor's, or Master's - in less than one year.</A>_______________________________________________<BR>openfabrics-ewg mailing list<BR>openfabrics-ewg@openib.org<BR>http://openib.org/mailman/listinfo/openfabrics-ewg<BR></DIV></BLOCKQUOTE><BR> <div> <HR SIZE=1> Sponsored Link <BR><BR>Talk more and pay less. Vonage can save you up to $300 a year on your phone bill. <A href="http://clk.atdmt.com/VON/go/yhxxxvon1080000017von/direct/01/">Sign up now.</A></BLOCKQUOTE><BR><p>
<hr size=1>Everyone is raving about <a href="http://us.rd.yahoo.com/evt=45083/*http://advision.webevents.yahoo.com/mailbeta">the all-new Yahoo! Mail beta.</a>