[Scst-devel] [ofa-general] WinOF_2_0_5/SRP initiator: slow reads and eventually hangs

[Bart Van Assche]

On Mon, Sep 21, 2009 at 5:56 PM, Chris Worley <worleys at gmail.com> wrote:
> On Mon, Sep 21, 2009 at 9:22 AM, Bart Van Assche
> <bart.vanassche at gmail.com> wrote:
>> Unless you will report the opposite I assume that the above issue (SRP
>> timeouts) has been solved by the solution I sent you via private
>> e-mail, namely to load the SRPT kernel module with the parameter
>> 'thread' set to one (modprobe ib_srpt thread=1).
>
> I do view that as a work-around, as it implies there is an issue in
> the threads... and multiple threads do provide more performance (which
> is what IB is all about).

This does not imply a threading issue in ib_srpt. And more threads do
not always provide higher performance. And you are misunderstanding
the effect of the ib_srpt kernel parameter called 'thread'. The effect
of this parameter is as follows:
* thread=0: as much as possible of the SRP protocol is processed in IB
interrupt context.
* thread=1: a kernel thread handles all SRP protocol processing. Work
is delegated from IB interrupt context to the SRPT kernel thread via a
queue (see also the srpt_completion() function in file ib_srpt.c).

So the kernel parameter 'thread' of ib_srpt allows to choose between
two significantly different behaviors of ib_srpt.

My hypothesis is that in your setup running ib_srpt with thread=0
resulted in SRPT's completion queue handler (srpt_completion()), which
keeps running as long as more completion queue elements can be
processed, took up too much time, and that this finally resulted in
remote SRP initiator disconnects.

Bart.