[openfabrics-ewg] Announcing the release of MVAPICH2 0.9.8 with Checkpoint/Restart, iWARP, RDMA CM-based connection manageme
david elsen
elsen_david at yahoo.com
Mon Nov 13 09:30:09 PST 2006
Sundeep,
I am sorry. I did not make myself very clear in my questions.
I have got the iWARP running on my set-up for the Ammasso and Chelsio cards.
To do this, I downloaded the iWARP code from the gen2 branch.
Now I was trying to run the OSC MPI tools MVAPICH2 there to run the iWARP traffic in my set-up.
When I am trying to build the MVAPICH2 0.9.8 tool, it has some mandatory variable called, OPEN_IB_HOME = /usr/local/ofed in the makefile make.mvapich2.iwarp, the build file which is to be run for the iWARP code.
My question is why should I have this reference to /usr/local/ofed there if I do not need to download the OFED distribution code to run the iWARP.
Is it possible to add more information in your MVAPICH2 0.9.8 User Guide describing how to build this and what are the dependencies?
David
Sundeep Narravula <narravul at cse.ohio-state.edu> wrote:
Hi David,
> iWARP is actually a part of the Open Fabrics SVN. It is available from
> a different branch.
>
> I am cc'ing this note to my group. One of the students (Sundeep) will
> send you the detailed instructions on which branch of OF to download
> and use.
The instructions for setting up iwarp on OpenFrabics is avalable on the
openIB wiki at
https://openib.org/tiki/tiki-index.php?page=Install+OpenIB+for+Chelsio+T3
https://openib.org/tiki/tiki-index.php?page=Install+OpenIB+for+Ammasso1100
Further, the branch you can download from the svn is
https://openib.org/svn/gen2/branches/iwarp/tags/iwarp-2.6.17-stable
Please let us know if you have any further questions.
Regards,
--Sundeep.
> > Is there any document describing the build process of the MVAPICH2
> > tool? I am going through the MVAPICH2 0.9.8 users guide and that
> > does not seem to be giving me the detailed information.
>
> We will add this additional information on our user guide.
>
> > Can you please provide README files for the iWARP which describes
> > TODO steps?
>
> Sundeep's information will help. If you have any additional questions,
> please feel free to ask us.
>
> Thanks,
>
> DK
>
> > Thanks,
> > David
> >
> > Dhabaleswar Panda
wrote:
> > The MVAPICH team is pleased to announce the availability of MVAPICH2
> > 0.9.8 with the following NEW features:
> >
> > - Checkpoint/Restart support for application transparent systems-level
> > fault tolerance. BLCR-based support using native InfiniBand Gen2
> > interface is provided. Flexible interface to work with different
> > file systems. Tested with ext3 (local disk), NFS and PVFS2.
> >
> > Performance of sample applications with checkpoint-restart using
> > PVFS2 and Lustre can be found here:
> >
> > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/performance/mvapich2/application/MVAPICH2-ckpt.html
> >
> > - iWARP support: Incorporates the support for OpenFabrics/Gen2-iWARP.
> > Tested with Chelsio T3 (10GigE) and Ammasso iWARP adapters and
> > drivers.
> >
> > - RDMA CM-based Connection management support
> >
> > - Shared memory optimizations for collective communication operations.
> > Efficient algorithms and optimizations for barrier, reduce and
> > all-reduce operations. Exploits the multi-core optimized shared
> > memory point-to-point communication support introduced in MVAPICH2
> > 0.9.6.
> >
> > Performance of sample collective operations with this new feature
> > can be found here:
> >
> > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/perf-coll.html
> >
> > - uDAPL support for NetEffect 10GigE adapter. Tested with
> > NetEffect NE010 adapter.
> >
> > More details on all features and supported platforms can be obtained
> > by visiting the following URL:
> >
> > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/mvapich2_features.html
> >
> > MVAPICH2 0.9.8 release is tested with the latest OFED 1.1 stack. It
> > continues to deliver excellent performance. Sample performance
> > numbers include:
> >
> > - OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR:
> > Two-sided operations:
> > - 2.81 microsec one-way latency (4 bytes)
> > - 1561 MB/sec unidirectional bandwidth
> > - 2935 MB/sec bidirectional bandwidth
> >
> > One-sided operations:
> > - 4.92 microsec Put latency
> > - 1569 MB/sec unidirectional Put bandwidth
> > - 2935 MB/sec bidirectional Put bandwidth
> >
> > - OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR (Dual-rail):
> > Two-sided operations:
> > - 2.81 microsec one-way latency (4 bytes)
> > - 3127 MB/sec unidirectional bandwidth
> > - 5917 MB/sec bidirectional bandwidth
> >
> > One-sided operations:
> > - 4.37 microsec Put latency
> > - 3137 MB/sec unidirectional Put bandwidth
> > - 5917 MB/sec bidirectional Put bandwidth
> >
> > - OpenFabrics/Gen2 on Opteron single-core with PCI-Ex and IBA-DDR:
> > Two-sided operations:
> > - 3.01 microsec one-way latency (4 bytes)
> > - 1402 MB/sec unidirectional bandwidth
> > - 2238 MB/sec bidirectional bandwidth
> >
> > One-sided operations:
> > - 4.65 microsec Put latency
> > - 1402 MB/sec unidirectional Put bandwidth
> > - 2238 MB/sec bidirectional Put bandwidth
> >
> > Performance numbers for all other platforms, system configurations and
> > operations can be viewed by visiting `Performance' section of the
> > project's web page.
> >
> > With the ADI-3-level design, MVAPICH2 0.9.8 delivers similar
> > performance for two-sided operations compared to MVAPICH 0.9.8.
> > Organizations and users interested in getting the best performance for
> > both two-sided and one-sided operations and also want to exploit
> > advanced features (such as fault tolerance with checkpoint/restart,
> > iWARP, RDMA CM connection management, multi-threading, integrated
> > multi-rail, multi-core optimization, memory hook support and optimized
> > collectives) may migrate from MVAPICH code base to MVAPICH2 code base.
> >
> > For downloading MVAPICH2 0.9.8 package and accessing the anonymous
> > SVN, please visit the following URL:
> >
> > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/
> >
> > A stripped down version of this release is also available at the
> > OpenFabrics SVN.
> >
> > All feedbacks, including bug reports and hints for performance tuning,
> > are welcome. Please post it to the mvapich-discuss mailing list.
> >
> > Thanks,
> >
> > MVAPICH Team at OSU/NBCL
> >
> > ======================================================================
> > MVAPICH/MVAPICH2 project is currently supported with funding from
> > U.S. National Science Foundation, U.S. DOE Office of Science,
> > Mellanox, Intel, Cisco Systems, Sun Microsystems and Linux Networx;
> > and with equipment support from Advanced Clustering, AMD, Apple,
> > Appro, Dell, IBM, Intel, Mellanox, Microway, PathScale, SilverStorm
> > and Sun Microsystems. Other technology partner includes Etnus.
> > ======================================================================
> >
> > _______________________________________________
> > openfabrics-ewg mailing list
> > openfabrics-ewg at openib.org
> > http://openib.org/mailman/listinfo/openfabrics-ewg
> >
> >
> >
> >
> > ---------------------------------
> > Sponsored Link
> >
> > Try Netflix today! With plans starting at only $5.99 a month what are you waiting for?
> > --0-119921422-1163376356=:19890
> > Content-Type: text/html; charset=iso-8859-1
> > Content-Transfer-Encoding: 8bit
> >
> > I am trying to use the OSC MPI tool for the iWARP and quite new to the open fabrics tools.
If I know it correctly, iWARP is not yet part of the OFED release.
But the iWARP makefile has reference to ofed code. Is it really required.
Is there any document describing the build process of the MVAPICH2 tool? I am going through the MVAPICH2 0.9.8 users guide and that does not seem to be giving me the detailed information.
Can you please provide README files for the iWARP which describes TODO steps?
Thanks,
David
Dhabaleswar Panda <panda at cse.ohio-state.edu> wrote:
The MVAPICH team is pleased to announce the availability of MVAPICH2
0.9.8 with the following NEW features:
-
> > Checkpoint/Restart support for application transparent systems-level
fault tolerance. BLCR-based support using native InfiniBand Gen2
interface is provided. Flexible interface to work with different
file systems. Tested with ext3 (local disk), NFS and PVFS2.
Performance of sample applications with checkpoint-restart using
PVFS2 and Lustre can be found here:
http://nowlab.cse.ohio-state.edu/projects/mpi-iba/performance/mvapich2/application/MVAPICH2-ckpt.html
- iWARP support: Incorporates the support for OpenFabrics/Gen2-iWARP.
Tested with Chelsio T3 (10GigE) and Ammasso iWARP adapters and
drivers.
- RDMA CM-based Connection management support
- Shared memory optimizations for collective communication operations.
Efficient algorithms and optimizations for barrier, reduce and
all-reduce operations. Exploits the multi-core optimized shared
memory point-to-point communication support introduced in
> > MVAPICH2
0.9.6.
Performance of sample collective operations with this new feature
can be found here:
http://nowlab.cse.ohio-state.edu/projects/mpi-iba/perf-coll.html
- uDAPL support for NetEffect 10GigE adapter. Tested with
NetEffect NE010 adapter.
More details on all features and supported platforms can be obtained
by visiting the following URL:
http://nowlab.cse.ohio-state.edu/projects/mpi-iba/mvapich2_features.html
MVAPICH2 0.9.8 release is tested with the latest OFED 1.1 stack. It
continues to deliver excellent performance. Sample performance
numbers include:
- OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR:
Two-sided operations:
- 2.81 microsec one-way latency (4 bytes)
- 1561 MB/sec unidirectional bandwidth
- 2935 MB/sec bidirectional bandwidth
One-sided operations:
- 4.92 microsec Put latency
- 1569 MB/sec unidirectional Put bandwidth
- 2935 MB/sec
> > bidirectional Put bandwidth
- OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR (Dual-rail):
Two-sided operations:
- 2.81 microsec one-way latency (4 bytes)
- 3127 MB/sec unidirectional bandwidth
- 5917 MB/sec bidirectional bandwidth
One-sided operations:
- 4.37 microsec Put latency
- 3137 MB/sec unidirectional Put bandwidth
- 5917 MB/sec bidirectional Put bandwidth
- OpenFabrics/Gen2 on Opteron single-core with PCI-Ex and IBA-DDR:
Two-sided operations:
- 3.01 microsec one-way latency (4 bytes)
- 1402 MB/sec unidirectional bandwidth
- 2238 MB/sec bidirectional bandwidth
One-sided operations:
- 4.65 microsec Put latency
- 1402 MB/sec unidirectional Put bandwidth
- 2238 MB/sec bidirectional Put bandwidth
Performance numbers for all other platforms, system configurations and
operations can be viewed by visiting `Performance' section of the
project's web page.
With the
> > ADI-3-level design, MVAPICH2 0.9.8 delivers similar
performance for two-sided operations compared to MVAPICH 0.9.8.
Organizations and users interested in getting the best performance for
both two-sided and one-sided operations and also want to exploit
advanced features (such as fault tolerance with checkpoint/restart,
iWARP, RDMA CM connection management, multi-threading, integrated
multi-rail, multi-core optimization, memory hook support and optimized
collectives) may migrate from MVAPICH code base to MVAPICH2 code base.
For downloading MVAPICH2 0.9.8 package and accessing the anonymous
SVN, please visit the following URL:
http://nowlab.cse.ohio-state.edu/projects/mpi-iba/
A stripped down version of this release is also available at the
OpenFabrics SVN.
All feedbacks, including bug reports and hints for performance tuning,
are welcome. Please post it to the mvapich-discuss mailing list.
Thanks,
> >
MVAPICH Team at OSU/NBCL
======================================================================
MVAPICH/MVAPICH2 project is currently supported with funding from
U.S. National Science Foundation, U.S. DOE Office of Science,
Mellanox, Intel, Cisco Systems, Sun Microsystems and Linux Networx;
and with equipment support from Advanced Clustering, AMD, Apple,
Appro, Dell, IBM, Intel, Mellanox, Microway, PathScale, SilverStorm
and Sun Microsystems. Other technology partner includes Etnus.
======================================================================
_______________________________________________
openfabrics-ewg mailing list
openfabrics-ewg at openib.org
http://openib.org/mailman/listinfo/openfabrics-ewg
> >
> >
> >
---------------------------------
Sponsored Link
> > Try Netflix today! With plans starting at only $5.99 a month what are you waiting for?
> > --0-119921422-1163376356=:19890--
> >
>
---------------------------------
Sponsored Link
$200,000 mortgage for $660/mo - 30/15 yr fixed, reduce debt, home equity - Click now for info
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20061113/4152a62d/attachment.html>
More information about the ewg
mailing list