[openib-general] Repost: AIO SDP and ttcp.aio: Event errors

[Libor Michalek]

On Tue, Apr 12, 2005 at 09:08:15PM +0100, Steven Wooding wrote:
> Hi,
> 
> I have been putting ttcp.aio through its paces and have a few questions.
> 
> 1. When -l is larger than 131072 I get an Event error <-22> on the transmit
> side and no data to transferred. Changing values of -n and -a do not make
> any difference.

  The FMRs need to be sized at initialization time. The code currently
picks 128K as the size for the FMRs, and does not support an AIO operation
that would span multiple FMRs. If you want to try larger AIO operations
with the current code you will need to recompile SDP with a larger FMR
size, which is determined by the constant SDP_IOCB_SIZE_MAX in sdp_iocb.h
It's been a while since I've last tried this, if you try it and have
problems let me know.

> 2. When using a value of 1 for -a (so I suppose this is non-aio), I get an
> Event error of <-32> on the transmit side and an <-104> on the receiver end.
> Only some of the data is transferred.

  I'll look into this, I'm seeing a problem on longer runs myself. With a
value of 1 for -a it still uses aio, the value only means how many aio
operations can be outstanding at a given time. This just means that a
single buffer will be submitted for read/write and a new one will not
be submitted until that buffer's IO completes.

> 3. For future reference, where can I find out what these Event error codes
> mean to give me a glue of what's going wrong.

  The errors are errno values. I'll make a note to write up which errors
are possible and what they are likely to mean.

> 4. I sometimes see significant differences in the transfer speed reported on
> the transmit and receiver ends. Is one more right than the other?

  Are the wall clock times for the data transfers small, on the order
of a few seconds? How big of a wall clock time difference are you seeing?

-Libor