[Scst-devel] [ofa-general] WinOF_2_0_5/SRP initiator: slow reads and eventually hangs

[Chris Worley]

On Wed, Sep 16, 2009 at 1:03 AM, Bart Van Assche
<bart.vanassche at gmail.com> wrote:
> On Tue, Sep 15, 2009 at 10:51 PM, Chris Worley <worleys at gmail.com> wrote:
>> In lots of testing today, I've seen this panic twice on the Ubuntu 8.10 targets:
>>
>> [  330.155992] ib_srpt: disconnected session
>> 0x00247100000000460024710000000046 because a new SRP_LOGIN_REQ has
>> been received.
>
> The above message means that an initiator logged in, did not log out
> and logged in again. Has one of the initiator systems e.g. been power
> cycled while an SRP connection was active ?

Yes.  Once ib_srp is hung on one device, I re-login to get the device
and test again.  I can't log out of the previous, as it's hung... this
"hanging" is the issue I'm having.  When I re-login, I get a new
device... i.e. I hung /dev/sdc and re-login to get /dev/sdd... then
test that until it hangs.

Using the ramdisks makes the problem much easier to trigger, and
occasionally causes the panic, especially using:

fio --rw=randrw --bs=1k --numjobs=64 --iodepth=64 --sync=0 \
   --direct=1 --randrepeat=0  --group_reporting --ioengine=libaio \
   --filename=/dev/sdp --name=test --loops=10000 --runtime=1600 \
   --rwmixread=100

...on the initiator to cause it.

Chris
>
>> [  357.207046] ib_srpt: srpt_xmit_response: tag= 17 channel in bad state 2
>
> This means that an attempt was made by SRPT to transmit a response for
> a channel in the state 2 (disconnecting). This must be analyzed
> further, just like the bug report triggered from
> srpt_abort_scst_cmd().
>
> [ ... ]
>
>> [  411.636699] WARNING: at /root/scst/srpt/src/ib_srpt.c:924
>> srpt_abort_scst_cmd+0xac/0x160 [ib_srpt]()
>> ...
>
> This message has been triggered by the statement WARN_ON("unexpected
> cmd state"). It must be analyzed whether this is a consequence of what
> went wrong before or whether this is a separate issue.
>
> [ ... ]
>
>> This may have been due to low memory, as I was using most target
>> memory for the ramdisk.
>
> The kernel warning message in ib_srpt.c at line 924 should never be
> triggered, not even under low memory circumstances. I'll have a look
> at this anyway.
>
> Thanks for the detailed report.
>
> Bart.
>