[ewg] Re: [Stgt-devel] Update (Re: open iSCSI over iSER target RPM ...)

FUJITA Tomonori tomof at acm.org
Sat Feb 9 08:41:36 PST 2008


From: Joe Landman <landman at scalableinformatics.com>
Subject: [Stgt-devel] Update (Re: open iSCSI over iSER target RPM ...)
Date: Thu, 07 Feb 2008 11:05:03 -0500

> Update:
> 
> [root at woody etc]# dd if=/dev/zero of=/big/local.file bs=256k count=100000
> 100000+0 records in
> 100000+0 records out
> 26214400000 bytes (26 GB) copied, 58.7484 seconds, 446 MB/s
> 
> Better. I rebuilt OFED 1.2.5.5.  Are there specific recommended tuning 
> guides for iSER?  Backing store in this case are real disks, and we can 
> sink/source >750 MB/s on them, so I am not worried about disk IO 
> bottlenecks, more worried about bad config of iSCSI/iSER.
> 
> BTW:  the 2TB LUN limit I asked about is still here in this code.  Same 
> machines (initiator and target) used for SRP reported correct LUN sizes. 
>   Here we are using the -868 open-iscsi initiator, and the tgt RPM 
> announced.  I would like to dig into this.
> 
> This is what I am getting in dmesg for this iSER target:
> 
> iscsi: registered transport (tcp)
> iscsi: registered transport (iser)
> iser: iser_connect:connecting to: 10.2.1.2, port 0xbc0c
> iser: iser_cma_handler:event 0 conn ffff81024b9f69c0 id ffff810209748c00
> iser: iser_cma_handler:event 2 conn ffff81024b9f69c0 id ffff810209748c00
> iser: iser_create_ib_conn_res:setting conn ffff81024b9f69c0 cma_id 
> ffff810209748c00: fmr_pool ffff81024bfb32c0 qp ffff8101cb16d600
> iser: iser_cma_handler:event 9 conn ffff81024b9f69c0 id ffff810209748c00
> iser: iscsi_iser_ep_poll:ib conn ffff81024b9f69c0 rc = 1
> scsi13 : iSCSI Initiator over iSER, v.0.1
> iser: iscsi_iser_conn_bind:binding iscsi conn ffff81021b65fa90 to 
> iser_conn ffff81024b9f69c0
>    Vendor: IET       Model: Controller        Rev: 0001
>    Type:   RAID                               ANSI SCSI revision: 05
> scsi 13:0:0:0: Attached scsi generic sg2 type 12
>    Vendor: IET       Model: VIRTUAL-DISK      Rev: 0001
>    Type:   Direct-Access                      ANSI SCSI revision: 05
> sdc : very big device. try to use READ CAPACITY(16).
> sdc : READ CAPACITY(16) failed.
> sdc : status=1, message=00, host=0, driver=08
> sdc : use 0xffffffff as device size
> SCSI device sdc: 4294967296 512-byte hdwr sectors (2199023 MB)
> sdc: Write Protect is off
> sdc: Mode Sense: 79 00 00 08
> SCSI device sdc: drive cache: write back
> sdc : very big device. try to use READ CAPACITY(16).
> sdc : READ CAPACITY(16) failed.
> sdc : status=1, message=00, host=0, driver=08
> sdc : use 0xffffffff as device size
> SCSI device sdc: 4294967296 512-byte hdwr sectors (2199023 MB)
> sdc: Write Protect is off
> sdc: Mode Sense: 79 00 00 08
> SCSI device sdc: drive cache: write back
>   sdc: unknown partition table
> sd 13:0:0:1: Attached scsi disk sdc
> sd 13:0:0:1: Attached scsi generic sg3 type 0
> 
> 
> and this is what we get in SRP
> 
> scsi6 : SRP.T10:0008F104039862A4
>    Vendor: SCST_BIO  Model: vdisk0            Rev:  096
>    Type:   Direct-Access                      ANSI SCSI revision: 04
> sdc : very big device. try to use READ CAPACITY(16).
> SCSI device sdc: 12693355130 512-byte hdwr sectors (6498998 MB)
> sdc: Write Protect is off
> sdc: Mode Sense: 6b 00 10 08
> SCSI device sdc: drive cache: write back w/ FUA
> 
> 
> This looks suspiciously like a 2^32 limit somewhere.

Can you try the latest git tree
(65a3f8b0c14305aaee5bcaade569b40882e8dd88)? It works for me:

scsi3 : iSCSI Initiator over TCP/IP
scsi 3:0:0:0: RAID              IET      Controller       0001 PQ: 0 ANSI: 5
scsi 3:0:0:1: Direct-Access     IET      VIRTUAL-DISK     0001 PQ: 0 ANSI: 5
sd 3:0:0:1: [sdb] Very big device. Trying to use READ CAPACITY(16).
sd 3:0:0:1: [sdb] 12884901888 512-byte hardware sectors (6597070 MB)
sd 3:0:0:1: [sdb] Write Protect is off
sd 3:0:0:1: [sdb] Mode Sense: 79 00 00 08
sd 3:0:0:1: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA


Somehow I forgot to add READ_CAPACITY_16 support. I would appreciate
it if you could read and write on over 2TB position properly.


> Our exported device is
> 
> [root at jr1 ~]# parted /dev/sdb print
> 
> Model: Areca jrvs1 (scsi)
> Disk /dev/sdb: 6500GB
> Sector size (logical/physical): 512B/512B
> Partition Table: loop
> 
> Number  Start   End     Size    File system  Flags
>   1      0.00kB  6500GB  6500GB  xfs
> 
> 
> and this is what tgtadm reports
> 
> [root at jr1 ~]# tgtadm --lld iscsi --op show --mode target
> Target 1: iqn.2001-04.com.jr1-jackrabbit.small
>      System information:
>          Driver: iscsi
>          Status: running
>      I_T nexus information:
>          I_T nexus: 4
>              Initiator: iqn.1996-04.voltaire.com:01:dfa8888a3fd
>              Connection: 0
>                  RDMA IP Address: 10.2.1.1
>      LUN information:
>          LUN: 0
>              Type: controller
>              SCSI ID: deadbeaf1:0
>              SCSI SN: beaf10
>              Size: 0
>              Online: No
>              Poweron/Reset: Yes
>              Removable media: No
>              Backing store: No backing store
>          LUN: 1
>              Type: disk
>              SCSI ID: deadbeaf1:1
>              SCSI SN: beaf11
>              Size: 5T
>              Online: Yes
>              Poweron/Reset: No
>              Removable media: No
>              Backing store: /dev/sdb
>      Account information:
>      ACL information:
>          10.2.1.1
> 
> So it looks like the LUN 1 is approximately correct (5T ???) on the 
> target, and incorrect when the initiator asks for it.

I changed tgt to show the capacity like Linux does:


Target 1: iqn.2007-03:marks-vtl-tgt:tulip
    System information:
        Driver: iscsi
        Status: running
    I_T nexus information:
        I_T nexus: 1
            Initiator: iqn.2005-03.org.open-iscsi:d38a581f3318
            Connection: 0
                IP Address: 192.168.11.15
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: deadbeaf1:0
            SCSI SN: beaf10
            Size: 0 MB
            Online: Yes
            Poweron/Reset: Yes
            Removable media: No
            Backing store: No backing store
        LUN: 1
            Type: disk
            SCSI ID: deadbeaf1:1
            SCSI SN: beaf11
            Size: 6597070 MB
            Online: Yes
            Poweron/Reset: No
            Removable media: No
            Backing store: /dev/sde
    Account information:
    ACL information:
        ALL


> Please note that I have successfully used the full 6+TB as an iSCSI 
> target using the SCST-iscsi code, so I do know that the initiator works 
> correctly.
> 
> Is there a source RPM/tree for this target?

I guess that RedHat, SUSE, and OFED have tgt RPMs now so I think that
you can find something.



More information about the ewg mailing list