[ofw] RE: [Scst-devel] SRP and DMIO - Windows Dynamic Disk

Tue Mar 17 12:20:08 PDT 2009

> -----Original Message-----
> From: Vladislav Bolkhovitin [mailto:vst at vlnb.net] 
> Sent: Tuesday, March 17, 2009 11:57 AM
> To: Sufficool, Stanley
> Cc: ofw at lists.openfabrics.org; Vu Pham; scst-devel
> Subject: Re: [Scst-devel] SRP and DMIO - Windows Dynamic Disk
> 
> 
> Sufficool, Stanley, on 03/17/2009 01:09 AM wrote:
> >> -----Original Message-----
> >> From: Vladislav Bolkhovitin [mailto:vst at vlnb.net]
> >> Sent: Monday, March 16, 2009 12:29 PM
> >> To: Sufficool, Stanley
> >> Cc: ofw at lists.openfabrics.org; scst-devel; Vu Pham
> >> Subject: Re: [Scst-devel] SRP and DMIO - Windows Dynamic Disk
> >>
> >>
> >>
> >> Sufficool, Stanley, on 03/16/2009 09:53 PM wrote:
> >>>> -----Original Message-----
> >>>> From: Vladislav Bolkhovitin [mailto:vst at vlnb.net]
> >>>> Sent: Monday, March 16, 2009 10:16 AM
> >>>> To: Sufficool, Stanley
> >>>> Cc: ofw at lists.openfabrics.org; scst-devel; Vu Pham
> >>>> Subject: Re: [Scst-devel] SRP and DMIO - Windows Dynamic Disk
> >>>>
> >>>>
> >>>> Sufficool, Stanley, on 03/14/2009 12:19 AM wrote:
> >>>>> Using Windows SRP (WinOF_2-2030_wnet_64)  / Linux SCST+SRPT
> >>>> (svn 695)
> >>>>> with dynamic disks leads to some strange behavior when the
> >>>> initiator is
> >>>>> rebooted. All basic disks come back online just fine. The
> >>>> dynamic disks
> >>>>> show up multiple times under the same names in disk
> >> manager, all as
> >>>>> offline or unreadable. The basic disks show as 
> unreadable in disk
> >>>>> management even though they are connected and working fine. 
> >>>> I expect to
> >>>>> see the dynamic disks as offline since the IB and SRP
> >>>> devices come up
> >>>>> after DMIO. I can script to bring these disks online after boot.
> >>>>>  
> >>>>> If I perform a disk rescan in disk manager to try and 
> recover the
> >>>>> dynamic disks, then all my basic disks that were 
> >> previously working
> >>>>> disappear as well.
> >>>>>  
> >>>>> Event logs from the Windows LDM service are:
> >>>>>
> >>>>> *LDM Event: *INTERNAL Error - No valid disk belonging 
> to the disk 
> >>>>> group was found (C1000096).
> >>>>>
> >>>>> *LDM Event:* Unexpected failure. Error code: 2 at 0200001D 
> >>>>> <mailto:2 at 0200001D>
> >>>>>
> >>>>> Stopping then restarting the WinOF SRP Miniport Device 
> restores my 
> >>>>> basic disk, but only one dynamic disk comes back online (with 
> >>>>> failed redundancy). The following doesn't look right 
> either. Does
> >>>> this have to
> >>>>> do with the MPIO from the MS iSCSI initiator package?
> >>>>>
> >>>>> *PartMgr Event:* Disk 3 will not be used because it is 
> a redundant 
> >>>>> path for disk 2.
> >>>>>
> >>>>> Restarting the target with the initiator left online
> >> gives the same
> >>>>> result.
> >>>>>
> >>>>> *PartMgr Event: *Disk 3 will not be used because it is 
> a redundant 
> >>>>> path for disk 2.
> >>>>>
> >>>>> I have attached the SCST messages from the target using the
> >>>> debug and
> >>>>> extracheck build of SCST.
> >>>>> I have the target setup for the dynamic disks as 500MB 
> vdisk file 
> >>>>> targets with 512 block sizes.
> >>>>>  
> >>>>> BTW: This works just fine using the SCST + iSCSI target
> >>>> with MS iSCSI
> >>>>> initiator. Maybe because the iSCSI initiator timing issues
> >>>> put it way
> >>>>> farther back in the load process behind the network stack
> >>>> and DMIO/LDM
> >>>>> service.
> >>>> I don't see anything wrong in the SCST log. According to
> >> it initiator
> >>>> only 2 times open/closed session (restarted?). If you enable SCSI
> >>>> logging, we can see all going SCSI commands and can say more.
> >>> echo "all" > trace_level
> >>>
> >>> Please see attached message. Hope this helps locate the issue.
> >> Stanley, sending 4+MB uncompressed attachments to public
> >> mailing lists 
> >> (2 of them!) is a *VERY* bad practice, please don't do that 
> >> in the future.
> > 
> > I will make sure that all large attachments are sent zipped and/or 
> > privately in the future. My sincere apologies.
> > 
> >> I checked the log in it's fine. So, there are no errors in 
> SCST core.
> >>
> >>>> Since iSCSI target works fine, then the issue must be
> >> either with SRP
> >>>> target, or SRP initiator, or Windows setup. For instance,
> >>>> Windows after 
> >>>> restart doesn't recognize the target's devices as the same 
> >> those were
> >>>> before. Hopefully, Vu can comment on it better.
> >>>>
> >>>> Do you use MPIO? Windows for MPIO needs a transport
> >> specific DSM. MS
> >>>> iSCSI initiator package has it, but how about Windows SRP
> >> initiator?
> >>> I am not explicitly using DMIO, however there are 2 posrt
> >> and 2 paths
> >>> to the target. If SRP detects this without configuration 
> then maybe.
> >> Then you are required to have an SRP DSM.
> > 
> > Well, no SRP DSM exists for Windows 2003 AFAIK. Many of the 
> MS forums 
> > dead end at "no DSM except iSCSI for W2k3".
> 
> MS has very strange DSM creation policy. Looks like they want 
> to charge 
> royalty from all non-iSCSI storage vendors.
> 
> > Is this something that the OFED-W would be working on?
> > 
> > Also, would there be a way to mask off the second port at 
> the target 
> > so that it wouldn't see the redundant path?
> 
> Physically disconnect another port? Use SCST access control to forbid 
> Windows to see devices from another port?

I would use SCST access controls to fix this, but as stated below, this 
can't be achieved when the Infiniband port ids are excluded from the
session name.

If I had individual HCA's, this would not be a problem. However, we are
working with
dual ports on one HCA.

> 
> > This was
> > achievable in the past with the <port id>+<port id> 
> initiator masking in
> > the SRP/SCST session id. But now it is <HCA id>+<HCA id>
> > So we get connects on all ports from the same 2 HCA's.
> > 
> >>>> Vlad
> >>>>
> >>>>
> >>
> > 
> > 
> ----------------------------------------------------------------------
> > --------
> > Apps built with the Adobe(R) Flex(R) framework and Flex 
> Builder(TM) are
> > powering Web 2.0 with engaging, cross-platform 
> capabilities. Quickly and
> > easily build your RIAs with Flex Builder, the 
> Eclipse(TM)based development
> > software that enables intelligent coding and step-through debugging.
> > Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> > _______________________________________________
> > Scst-devel mailing list
> > Scst-devel at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/scst-devel
> > 
> 
> 