[ewg] Re: [Stgt-devel] Update (Re: open iSCSI over iSER target RPM ...)
FUJITA Tomonori
tomof at acm.org
Sat Feb 9 08:41:36 PST 2008
From: Joe Landman <landman at scalableinformatics.com>
Subject: [Stgt-devel] Update (Re: open iSCSI over iSER target RPM ...)
Date: Thu, 07 Feb 2008 11:05:03 -0500
> Update:
>
> [root at woody etc]# dd if=/dev/zero of=/big/local.file bs=256k count=100000
> 100000+0 records in
> 100000+0 records out
> 26214400000 bytes (26 GB) copied, 58.7484 seconds, 446 MB/s
>
> Better. I rebuilt OFED 1.2.5.5. Are there specific recommended tuning
> guides for iSER? Backing store in this case are real disks, and we can
> sink/source >750 MB/s on them, so I am not worried about disk IO
> bottlenecks, more worried about bad config of iSCSI/iSER.
>
> BTW: the 2TB LUN limit I asked about is still here in this code. Same
> machines (initiator and target) used for SRP reported correct LUN sizes.
> Here we are using the -868 open-iscsi initiator, and the tgt RPM
> announced. I would like to dig into this.
>
> This is what I am getting in dmesg for this iSER target:
>
> iscsi: registered transport (tcp)
> iscsi: registered transport (iser)
> iser: iser_connect:connecting to: 10.2.1.2, port 0xbc0c
> iser: iser_cma_handler:event 0 conn ffff81024b9f69c0 id ffff810209748c00
> iser: iser_cma_handler:event 2 conn ffff81024b9f69c0 id ffff810209748c00
> iser: iser_create_ib_conn_res:setting conn ffff81024b9f69c0 cma_id
> ffff810209748c00: fmr_pool ffff81024bfb32c0 qp ffff8101cb16d600
> iser: iser_cma_handler:event 9 conn ffff81024b9f69c0 id ffff810209748c00
> iser: iscsi_iser_ep_poll:ib conn ffff81024b9f69c0 rc = 1
> scsi13 : iSCSI Initiator over iSER, v.0.1
> iser: iscsi_iser_conn_bind:binding iscsi conn ffff81021b65fa90 to
> iser_conn ffff81024b9f69c0
> Vendor: IET Model: Controller Rev: 0001
> Type: RAID ANSI SCSI revision: 05
> scsi 13:0:0:0: Attached scsi generic sg2 type 12
> Vendor: IET Model: VIRTUAL-DISK Rev: 0001
> Type: Direct-Access ANSI SCSI revision: 05
> sdc : very big device. try to use READ CAPACITY(16).
> sdc : READ CAPACITY(16) failed.
> sdc : status=1, message=00, host=0, driver=08
> sdc : use 0xffffffff as device size
> SCSI device sdc: 4294967296 512-byte hdwr sectors (2199023 MB)
> sdc: Write Protect is off
> sdc: Mode Sense: 79 00 00 08
> SCSI device sdc: drive cache: write back
> sdc : very big device. try to use READ CAPACITY(16).
> sdc : READ CAPACITY(16) failed.
> sdc : status=1, message=00, host=0, driver=08
> sdc : use 0xffffffff as device size
> SCSI device sdc: 4294967296 512-byte hdwr sectors (2199023 MB)
> sdc: Write Protect is off
> sdc: Mode Sense: 79 00 00 08
> SCSI device sdc: drive cache: write back
> sdc: unknown partition table
> sd 13:0:0:1: Attached scsi disk sdc
> sd 13:0:0:1: Attached scsi generic sg3 type 0
>
>
> and this is what we get in SRP
>
> scsi6 : SRP.T10:0008F104039862A4
> Vendor: SCST_BIO Model: vdisk0 Rev: 096
> Type: Direct-Access ANSI SCSI revision: 04
> sdc : very big device. try to use READ CAPACITY(16).
> SCSI device sdc: 12693355130 512-byte hdwr sectors (6498998 MB)
> sdc: Write Protect is off
> sdc: Mode Sense: 6b 00 10 08
> SCSI device sdc: drive cache: write back w/ FUA
>
>
> This looks suspiciously like a 2^32 limit somewhere.
Can you try the latest git tree
(65a3f8b0c14305aaee5bcaade569b40882e8dd88)? It works for me:
scsi3 : iSCSI Initiator over TCP/IP
scsi 3:0:0:0: RAID IET Controller 0001 PQ: 0 ANSI: 5
scsi 3:0:0:1: Direct-Access IET VIRTUAL-DISK 0001 PQ: 0 ANSI: 5
sd 3:0:0:1: [sdb] Very big device. Trying to use READ CAPACITY(16).
sd 3:0:0:1: [sdb] 12884901888 512-byte hardware sectors (6597070 MB)
sd 3:0:0:1: [sdb] Write Protect is off
sd 3:0:0:1: [sdb] Mode Sense: 79 00 00 08
sd 3:0:0:1: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Somehow I forgot to add READ_CAPACITY_16 support. I would appreciate
it if you could read and write on over 2TB position properly.
> Our exported device is
>
> [root at jr1 ~]# parted /dev/sdb print
>
> Model: Areca jrvs1 (scsi)
> Disk /dev/sdb: 6500GB
> Sector size (logical/physical): 512B/512B
> Partition Table: loop
>
> Number Start End Size File system Flags
> 1 0.00kB 6500GB 6500GB xfs
>
>
> and this is what tgtadm reports
>
> [root at jr1 ~]# tgtadm --lld iscsi --op show --mode target
> Target 1: iqn.2001-04.com.jr1-jackrabbit.small
> System information:
> Driver: iscsi
> Status: running
> I_T nexus information:
> I_T nexus: 4
> Initiator: iqn.1996-04.voltaire.com:01:dfa8888a3fd
> Connection: 0
> RDMA IP Address: 10.2.1.1
> LUN information:
> LUN: 0
> Type: controller
> SCSI ID: deadbeaf1:0
> SCSI SN: beaf10
> Size: 0
> Online: No
> Poweron/Reset: Yes
> Removable media: No
> Backing store: No backing store
> LUN: 1
> Type: disk
> SCSI ID: deadbeaf1:1
> SCSI SN: beaf11
> Size: 5T
> Online: Yes
> Poweron/Reset: No
> Removable media: No
> Backing store: /dev/sdb
> Account information:
> ACL information:
> 10.2.1.1
>
> So it looks like the LUN 1 is approximately correct (5T ???) on the
> target, and incorrect when the initiator asks for it.
I changed tgt to show the capacity like Linux does:
Target 1: iqn.2007-03:marks-vtl-tgt:tulip
System information:
Driver: iscsi
Status: running
I_T nexus information:
I_T nexus: 1
Initiator: iqn.2005-03.org.open-iscsi:d38a581f3318
Connection: 0
IP Address: 192.168.11.15
LUN information:
LUN: 0
Type: controller
SCSI ID: deadbeaf1:0
SCSI SN: beaf10
Size: 0 MB
Online: Yes
Poweron/Reset: Yes
Removable media: No
Backing store: No backing store
LUN: 1
Type: disk
SCSI ID: deadbeaf1:1
SCSI SN: beaf11
Size: 6597070 MB
Online: Yes
Poweron/Reset: No
Removable media: No
Backing store: /dev/sde
Account information:
ACL information:
ALL
> Please note that I have successfully used the full 6+TB as an iSCSI
> target using the SCST-iscsi code, so I do know that the initiator works
> correctly.
>
> Is there a source RPM/tree for this target?
I guess that RedHat, SUSE, and OFED have tgt RPMs now so I think that
you can find something.
More information about the ewg
mailing list