[openib-general] [RFC] [PATCH] ib_cm: send DREP in response to unmatched DREQ

[Or Gerlitz]

On 9/28/06, Sean Hefty <mshefty at ichips.intel.com> wrote:
> Or Gerlitz wrote:
> > My understanding is that without this patch the side that sends the DREQ
> > would do few DREQ resends as of the "firsts" DREPs being lost and no
> > DREPs sent once the id at the peer side left the timewait state, correct?
>
> This is correct.  Note that the number of DREQ retries was changed to 15 now.

do you mean internally changed in the CM or somehow controlled from
the outside by uDAPL?

> > Can you please share what were the implications with intel MPI running a
> > 64 nodes (128 ranks?) job? was the issue here just making the ***job
> > termination time*** bigger?
>
> The job termination time was taking about a minute waiting for the DREQ to
> timeout.  When running a series of tests, this becomes a fairly large issue.

Just something you might want to verify with the intel MPI  team, does
their terminate code looks like:

for (i=0,N-1)
  call dat_ep_disconnect(ep[i]...)

j=0
while(j < N) {
   dat_evd_wait(conn_evd)
   verify its a disconnected event on EP[i] for some 0 < i < N-1
   j++
 }

and not

for (i=0,N-1)
   dat_ep_disconnect(ep[i]...)
   dat_evd_wait(conn_evd)
}

Or.