[ofa-general] Re: [PATCH RFC] libibumad: give up cpu time slice if write() falied

[Sasha Khapyorsky]

Hi Yevgeny,

On 17:26 Thu 29 Jan     , Yevgeny Kliteynik wrote:
> 
> While running opensm on a single-core CPU I've noticed the following problem:
> when SA is stressed with many SA queries (such as when you run "osmtest -f f"
> on a multi-core CPU machine), sometimes opensm fails to send responses.
> It appears that send buffer gets full, and write() fails.

What is an errno?

> Since the CPU has a single core, when the sender thread retries to send the
> same packet, it fails again and again, because the driver didn't have the chance
> to transmit something from the send buffer.

Why not? DMA doesn't require CPU cycles. And even if it is not DMA then
an user space running thread will be rescheduled some days.

> Then I added 20 msec sleep after write() failure to make it give up the cpu time
> slice, and I saw some improvement. When I added sleep of several seconds, the problem
> disappeared completely, but then of course, the client's transaction will fail with
> timeout unless you specifically increase the transaction timeout on the client side.
> 
> So, what do you think about the following patch?

If errno is EAGAIN (and I would expect that it is) then you can handle
this in an upper layer application (if it is relevant at all), and
limiting library is not a good idea IMO. For example how I would make
stress-test then or benchmark? And what is the meaning of this "magic"
number 20 ms?

Sasha