[ofw] RE: SRP Multipath LUN Masking
Sufficool, Stanley
ssufficool at sbcounty.gov
Fri Sep 4 14:15:23 PDT 2009
> -----Original Message-----
> From: Vladislav Bolkhovitin [mailto:vst at vlnb.net]
> Sent: Friday, September 04, 2009 9:00 AM
> To: Sufficool, Stanley
> Cc: scst-devel
> Subject: Re: [Scst-devel] iSCSI > 2TB Fails
>
>
> Sufficool, Stanley, on 09/03/2009 10:41 PM wrote:
> >
> >> -----Original Message-----
> >> From: Vladislav Bolkhovitin [mailto:vst at vlnb.net]
> >> Sent: Thursday, September 03, 2009 10:11 AM
> >> To: Sufficool, Stanley
> >> Cc: scst-devel
> >> Subject: Re: [Scst-devel] iSCSI > 2TB Fails
> >>
> >>
> >> Sufficool, Stanley, on 08/31/2009 08:17 PM wrote:
> >>>>>> I did confirm that changing the block size on the target
> >>>>> from 512 to
> >>>>>> 4096 fixes the issue for Windows MS iSCSI initiators. I
> >>>>> fairly certain
> >>>>>> that this is a wrap around issue.
> >>>>> Not necessary. ISCSI-SCST has many improvements, which can badly
> >>>>> affect not preparing to them (i.e. faulty) initiators.
> >> For instance,
> >>>>> IET sends
> >>>>> in reply to READ CAPACITY(16) command only 12 bytes,
> >> although this
> >>>>> command supposed to have 32 bytes reply. In the missed by
> >> IET bytes
> >>>>> ISCSI-SCST's vdisk tells to initiator amount of physical
> >> blocks per
> >>>>> logical block which the target has. Since Linux always uses
> >>>>> 4K internal
> >>>>> handling units (pages), with 512 bytes blocks there are 8
> >>>>> logical blocks
> >>>>> per target's physical block. This information allows
> >> initiators to
> >>>>> optimize performance, e.g., by avoiding read-modify-write
> >> sequences
> >>>>> necessary with not 4K aligned writes. Try with the attached
> >>>>> patch, which
> >>>>> disables that functionality.
> >>>>>
> >>>>> I've heard that the issue you are experiencing can be
> specific to
> >>>>> some versions (builds?) of Windows. Some particular
> version/build
> >>>>> of Windows
> >>>>> from some MSDN issue has it, but other both earlier and later
> >>>>> versions/builds don't. I've got no idea what it can mean.
> >>>> Perhaps, MS
> >>>>> issued a hot fix for that.
> >>>> Note on Max Size of NTFS:
> >>>> 2^32 clusters minus 1 cluster
> >>>> Using a 64-kilobyte (KB) cluster (the maximum NTFS cluster
> >>>> size) the maximum size of an NTFS volume is 256 TB minus 64 KB.
> >>>> Using a 4-KB cluster (the default NTFS cluster size),
> >>>> the maximum size of an NTFS volume is 16 TB minus 4 KB.
> >>>> Using 512-B, 2TB - 512 bytes?
> >>>>
> >>>> Given the above, this would have never worked anyhow.
> >>>>
> >>>> In looking at our production systems, I see that we user
> 4KB on our
> >>>> 2+ TB volumes, so this is a moot issue. Forgive the noise.
> >>>
> >>> Reducing the READ_CAP16_Len to 12 corrected the issue for MS iSCSI
> >>> initiator.
> >> So, do you confirm that applying the patch fixes your problem?
> >
> > Yes it allows the 2.4 TB lun with 512 byte sector size to be
> > recognized by MS iSCSI 2.08.
> >
> >> Then this is exactly as I suspected: your version of Windows has
> >> problems processing full 32 bytes replies to READ
> >> CAPACITY(16), which is
> >> fixed in later versions (by some hot fixes?). This is in line
> >> with what
> >> I know that not all versions of Windows affected by that.
> >
> > I cannot find any reference to a hotfix for MS iSCSI 2.08
> on Windows
> > 2003 SR2 to address this issue. I have search over the last
> few days
> > now.
> >
> >>> I noted that the kernel drivers/scsi/sd.c parses the READ_CAP16
> >>> response as 8 byte LBA and 4 byte sector count, leaving
> off bit 13
> >>> that you set in the iscsi response. Is this why it is working on
> >>> Linux?
> >> The SCST implementation is fully confirmed to SBC, at least I
> >> don't know
> >> any problems in it.
> >
> > I do not pretend to know much about the SCSI SBC-2 command set and
> > ANSI requires a purchase to get the SBC-2 final standard.
> This is what
> > I found on the response for READ CAPACITY (16)
> > http://www.t10.org/ftp/t10/document.01/01-246r1.pdf
> > Page 4:
> > Table 5 - Long Read Capacity Data
> > Bytes 0-7 = LBA Address
> > Bytes 9-11 = Block length in bytes
> > ***No further bytes defined
> >
> > Do you have another reference source for the SCSI SBC-2 command set?
> >
> > I cannot find any initiator or SCSI layer that utilizes
> read capacity
> > (16) response bytes above byte 11.
> > linux-2.6.30/drivers/scsi/sd.c defined RC16_LEN = 32,
> but only bytes
> > 0-11 used @ line 1348
> >
> > So is this a double fault since MS iSCSI 2.08 should ignore bytes
> > above 11 since they are not defined in SBC-2 and SCST
> should not set
> > them?
>
> At first, you're using pretty old version of SBC from 2001
> year. Get a
> newer version from, e.g.,
> http://web.archive.org/web/*/http://www.t10.org/ftp/t10/drafts
> /, and you
> will see that READ CAPACITY(16) has 32 bytes response. Current Linux
> uses additional bytes from the response, see
sd_read_protection_type().
> At second, if Windows doesn't understand >12 bytes reply, it doesn't
> have to ask for more bytes. It can ask 12 bytes only and be happy.
> Asking for more bytes and then blame target for returning them is
> something pretty much insane.
>
> But you reminded me that SCST vdisk still claims itself as SCSI-2,
> although for a long time has been developed as SCSI-3. Thanks for
that.
> I fixed it. Now vdisk is SCSI-3.
>
> BTW, some time ago Bart implemented for you the SRP nodes addressing
you
> asked (use_port_guid_in_session_name kernel parameter). But we have no
> feedback from you if it helped you or not. This is a hack, so, if you
in
> the next few weeks don't confirm that it works, we will remove it.
>
I finally just unplugged one of the two ports on the target and
initiator to get connection.
ib_srpt: received SRP_LOGIN_REQ with i_port_id
0x1a4bffff0cd044:0x19bbffff00d7c8, t_port_id
0x1a4bffff0cd044:0x1a4bffff0cd044 and length 996 on port 2 >
(guid=0xfe80000000000000:0x1a4bffff0cd046)
scst: Using security group "nas02" for initiator
"0x001a4bffff0cd0440019bbffff00d7c8"
INITIATOR
-------------------------------------------
DEFINE_NODE
lid 0xC
base_version 0x1
class_version 0x1
node_type 0x1 # (Channel Adapter)
num_ports 0x2
sys_guid 0x0019bbffff00d7cb
node_guid 0x0019bbffff00d7c8
port_guid 0x0019bbffff00d7c9
partition_cap 0x40
device_id 0x6278
revision 0xA0
# port_num 0x1
TARGET
-------------------------------------------
DEFINE_NODE
lid 0x9
base_version 0x1
class_version 0x1
node_type 0x1 # (Channel Adapter)
num_ports 0x2
sys_guid 0x001a4bffff0cd047
node_guid 0x001a4bffff0cd044
port_guid 0x001a4bffff0cd046
partition_cap 0x40
device_id 0x6278
revision 0xA0
# port_num 0x2
>From the above, you can still see that the target is advertizing itself
on the fabric under one GUID which is the GUID for the node
(0x1a4bffff0cd044) instead of the individual port (0x1a4bffff0cd045 or
0x1a4bffff0cd046). The initiator is also using the node guid. This means
there is no way to mask off the lun on redundant paths. Each node has 2
ports identified by a port guid. I need to get the target to advertize
on 2 distinct port guids to give 2 distinct named paths to the target.
I'll work on this internally as I have time. But the official answer
from WinOFED seems to be "no multiport/multipath configuration is
supported on Windows SRP".
> Vlad
More information about the ofw
mailing list