[Scst-devel] [ofa-general] WinOF_2_0_5/SRP initiator: slow reads and eventually hangs

[Vladislav Bolkhovitin]

Chris Worley, on 09/15/2009 08:53 PM wrote:
> On Tue, Sep 15, 2009 at 10:43 AM, Vladislav Bolkhovitin <vst at vlnb.net> wrote:
>> Chris Worley, on 09/15/2009 07:50 PM wrote:
>>> On Tue, Sep 15, 2009 at 12:10 AM, Bart Van Assche
>>> <bart.vanassche at gmail.com> wrote:
>>>> On Tue, Sep 15, 2009 at 1:03 AM, Chris Worley <worleys at gmail.com> wrote:
>>>>> On Mon, Sep 14, 2009 at 12:51 PM, Vladislav Bolkhovitin <vst at vlnb.net>
>>>>> wrote:
>>>>>> Chris Worley, on 09/11/2009 11:50 PM wrote:
>>>>>>> I've definitely removed the switch/firmware from being the cause.
>>>>>>>
>>>>>>> I'm thinking the reason you can't repeat the test may be latency
>>>>>>> related.  We get ~50usecs average latency (on small block sizes),
>>>>>>> which can't be achieved using regular SSD's (and rotating drives are
>>>>>>> nowhere close).  Maybe a ramdisk would help repeat the issue.
>>>>>> I think you should try to reproduce the problem with ramdisk or nullio.
>>>>>> By
>>>>>> so you will eliminate possible influence of the SSD backend.
>>>>> W/ 12GB RAM in the target, I created a 7GB ramdisk:
>>>>>
>>>>> mount -t ramfs -o size=7g ramfs /mnt/
>>>>> dd if=/dev/zero of=/mnt/foo bs=1024k count=7000
>>>>> echo "open ramdisk /mnt/foo" > /proc/scsi_tgt/vdisk/vdisk
>>>>> echo "add ramdisk 2" >/proc/scsi_tgt/groups/Default/devices
>>>>>
>>>>> Then, on the initiator, I tested it... and it hung during sequential
>>>>> 8KB block reads:
>>>>>
>>>>> fio --rw=read --bs=8k --numjobs=64 --iodepth=64 --sync=0 --direct=1
>>>>> --randrepeat=0 \
>>>>>  --group_reporting --ioengine=libaio --filename=/dev/sde --name=test
>>>>> --loops=10000 --runtime=600
>>>>>
>>>>> Note that I was running the SM on the target this time too.
>>>> Which Linux distro was installed on the inititiator and on the target
>>>> ? And if applicable, which OFED version ? Which kernel messages were
>>>> logged by SRPT around the time the issue occurred (after having
>>>> enabled SRPT logging first) ?
>>> As logging hadn't helped this issue previously, I've not been enabling
>>> it.  That plus the kernel hacks needed to invoke logging, it's not
>>> worth enabling.
>>>
>>> This was with Ubuntu 8.10, built-in IB on the 2.6.27-14-server kernel.
>>>
>>> I couldn't get ramdisks working w/ SCST in RHEL5.2.  When running:
>>>
>>> echo "open ramdisk /mnt/foo" > /proc/scsi_tgt/vdisk/vdisk
>>>
>>> I get the error:
>>>
>>> dev_vdisk: ***ERROR***: Wrong f_op or FS doesn't have required
>>> capabilities
>>>
>>> ... which doesn't occur in the Ubuntu kernel, so I've been unable to
>>> test RHEL kernels w/ ramdisks.  In general, this problem occurs w/ 8KB
>>> and smaller blocks w/ the Ubuntu kernels, and 2KB and smaller blocks
>>> w/ RHEL kernels.
>> Use ramfs instead.
> 
> Do you mean:
> 
> mount -t ramfs -o size=7g ramfs /mnt/

You should then create a file on it and use it.

> ?
> 
> That's what I'm doing.
> 
> Chris
>>> Chris
>>>> Bart.
>>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Come build with us! The BlackBerry® Developer Conference in SF, CA
>>> is the only developer event you need to attend this year. Jumpstart your
>>> developing skills, take BlackBerry mobile applications to market and stay
>>> ahead of the curve. Join us from November 9-12, 2009. Register now!
>>> http://p.sf.net/sfu/devconf
>>> _______________________________________________
>>> Scst-devel mailing list
>>> Scst-devel at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/scst-devel
>>>
>>
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>