[openfabrics-ewg] [openib-general] OFED 1.1-rc3 is ready

[Woodruff, Robert J]

 Tziporet Wrote,
>Hi,

>OFED 1.1-RC3 is available on 
https://openib.org/svn/gen2/branches/1.1/ofed/releases/
>File: OFED-1.1-rc3.tgz
>Please report any issues in bugzilla http://openib.org/bugzilla/

Hi all, I installed the RC3 package on my Xeon/Lindenhurst platforms
and with the pathscale card I have the following problem
when trying to run Intel MPI and NetPipe.
The is on a RedHat EL4-U3 (2.6.9-34EL kernel).
Has anyone else been able to run DAPL/RDMA programs like Intel MPI
over the Pathscale cards with OFED-1.1-RC3 ?


# List of Benchmarks to run:

# PingPong
# PingPing
# Sendrecv
# Exchange
# Allreduce
# Reduce
# Reduce_scatter
# Allgather
# Allgatherv
# Alltoall
# Bcast
# Barrier
[1][rdma_iba.c:260] Intel MPI fatal error: DTO operation completed with
error. status=0x8. cookie=0x514ed0
rank 1 in job 2  rkl-13-ib0_32788   caused collective abort of all ranks
  exit status of rank 1: return code 255 

I also tried a version of NetPipe that was modified to use DAPL and
it works for messages < 2048 and then hangs 

Now starting the main loop
  0:       1 bytes   1000 times -->      1.53 Mbps in       4.98 usec
  1:       2 bytes   1000 times -->      3.11 Mbps in       4.91 usec
  2:       3 bytes   1000 times -->      4.72 Mbps in       4.85 usec
  3:       4 bytes   1000 times -->      6.20 Mbps in       4.92 usec
  4:       6 bytes   1000 times -->      9.23 Mbps in       4.96 usec
  5:       8 bytes   1000 times -->     12.41 Mbps in       4.92 usec
  6:      12 bytes   1000 times -->     18.61 Mbps in       4.92 usec
  7:      13 bytes   1000 times -->     19.98 Mbps in       4.96 usec
  8:      16 bytes   1000 times -->     24.48 Mbps in       4.99 usec
  9:      19 bytes   1000 times -->     29.04 Mbps in       4.99 usec
 10:      21 bytes   1000 times -->     32.31 Mbps in       4.96 usec
 11:      24 bytes   1000 times -->     36.72 Mbps in       4.99 usec
 12:      27 bytes   1000 times -->     40.67 Mbps in       5.06 usec
 13:      29 bytes   1000 times -->     44.24 Mbps in       5.00 usec
 14:      32 bytes   1000 times -->     49.15 Mbps in       4.97 usec
 15:      35 bytes   1000 times -->     53.18 Mbps in       5.02 usec
 16:      45 bytes   1000 times -->     67.60 Mbps in       5.08 usec
 17:      48 bytes   1000 times -->     72.32 Mbps in       5.06 usec
 18:      51 bytes   1000 times -->     76.65 Mbps in       5.08 usec
 19:      61 bytes   1000 times -->     90.20 Mbps in       5.16 usec
 20:      64 bytes   1000 times -->     94.50 Mbps in       5.17 usec
 21:      67 bytes   1000 times -->     96.89 Mbps in       5.28 usec
 22:      93 bytes   1000 times -->    134.20 Mbps in       5.29 usec
 23:      96 bytes   1000 times -->    137.11 Mbps in       5.34 usec
 24:      99 bytes   1000 times -->    139.72 Mbps in       5.41 usec
 25:     125 bytes   1000 times -->    175.08 Mbps in       5.45 usec
 26:     128 bytes   1000 times -->    184.12 Mbps in       5.30 usec
 27:     131 bytes   1000 times -->    184.72 Mbps in       5.41 usec
 28:     189 bytes   1000 times -->    258.25 Mbps in       5.58 usec
 29:     192 bytes   1000 times -->    269.32 Mbps in       5.44 usec
 30:     195 bytes   1000 times -->    270.92 Mbps in       5.49 usec
 31:     253 bytes   1000 times -->    339.98 Mbps in       5.68 usec
 32:     256 bytes   1000 times -->    347.01 Mbps in       5.63 usec
 33:     259 bytes   1000 times -->    349.64 Mbps in       5.65 usec
 34:     381 bytes   1000 times -->    491.64 Mbps in       5.91 usec
 35:     384 bytes   1000 times -->    495.59 Mbps in       5.91 usec
 36:     387 bytes   1000 times -->    493.49 Mbps in       5.98 usec
 37:     509 bytes   1000 times -->    621.98 Mbps in       6.24 usec
 38:     512 bytes   1000 times -->    639.37 Mbps in       6.11 usec
 39:     515 bytes   1000 times -->    632.97 Mbps in       6.21 usec
 40:     765 bytes   1000 times -->    854.35 Mbps in       6.83 usec
 41:     768 bytes   1000 times -->    878.14 Mbps in       6.67 usec
 42:     771 bytes   1000 times -->    878.74 Mbps in       6.69 usec
 43:    1021 bytes   1000 times -->   1067.29 Mbps in       7.30 usec
 44:    1024 bytes   1000 times -->   1073.29 Mbps in       7.28 usec
 45:    1027 bytes   1000 times -->   1076.14 Mbps in       7.28 usec
 46:    1533 bytes   1000 times -->   1396.85 Mbps in       8.37 usec
 47:    1536 bytes   1000 times -->   1407.83 Mbps in       8.32 usec
 48:    1539 bytes   1000 times -->   1385.12 Mbps in       8.48 usec
 49:    2045 bytes   1000 times -->   1647.53 Mbps in       9.47 usec
 50:    2048 bytes   1000 times -->   1657.56 Mbps in       9.43 usec
 51:    2051 bytes   1000 times -->
<- hangs here