[ofa-general] RE: [ewg] Re: SRP HA dm_multipath testing and questions

[Chieng Etta]

Hi Moiz,

I tested "adding new storage" on both OFED 1.2-beta1 and OFED 1.2-rc1. 
I used srp_daemon.sh to discover and add new storage automatically.  

On OFED 1.2-beta1, the default "retries" value at srp_daemon.sh was set to
300 seconds and I changed it to 60 seconds. The initiator discovered the new
target right away, but it took a few minutes to add the new target and new
path.  

On OFED 1.2-rc1, I changed the "retries" value at srp_daemon.sh to 30
seconds.  The initiator discovered the new target, added target and added
path within 30 seconds. 

Thanks,
Etta

-----Original Message-----
From: Moiz Kohari [mailto:mkohari at novell.com] 
Sent: Friday, April 13, 2007 1:27 PM
To: 'Scott Weitzenkamp (sweitzen)'; 'Ishai Rabinovitz'; Chieng Etta
Cc: 'Roland Dreier (rdreier)'; ewg at lists.openfabrics.org; Ken L Johnson;
Moiz Kohari; 'openib'
Subject: RE: [ewg] Re: SRP HA dm_multipath testing and questions

Hi,

Discovery of new storage should not take multiple minutes, at least we
haven't seen this type of behavior.  How exactly are you adding the storage
(using ibsrpadm command)?  any idea where the delay is occuring, discovery
of SRP targets or adding targets to the system?

Thanks,
Moiz

>>> On 4/12/2007 at 10:37 AM, in message
<000c01c77d20$d2bd5f40$c801a8c0 at ettac>,
"Chieng Etta" <etta at systemfabricworks.com> wrote:
> I tried adding/removing new storage on sles10.  It took few minutes to
find
> the new target devices (the new target message was showed on
> /var/log/messages) then took few minutes to add the path. I did not run
> multipath again.  The srp_daemon.sh scanned the new target and added path
> automatically.  
> 
> Thanks,
> Etta
> 
> -----Original Message-----
> From: Scott Weitzenkamp (sweitzen) [mailto:sweitzen at cisco.com] 
> Sent: Wednesday, April 11, 2007 4:59 PM
> To: Ishai Rabinovitz; Chieng Etta
> Cc: Roland Dreier (rdreier); ewg at lists.openfabrics.org; openib;
> mkohari at novell.com 
> Subject: RE: [ewg] Re: SRP HA dm_multipath testing and questions
> 
> I haven't tried adding or removing storage, just failover.  I guess
> leave 91-srp.rules in for now, it seems benign.
> 
> Scott 
> 
>> -----Original Message-----
>> From: Ishai Rabinovitz [mailto:ishai at dev.mellanox.co.il] 
>> Sent: Tuesday, April 10, 2007 9:46 PM
>> To: Chieng Etta
>> Cc: Scott Weitzenkamp (sweitzen); Roland Dreier (rdreier); 
>> ewg at lists.openfabrics.org; 'openib'; mkohari at novell.com 
>> Subject: Re: [ewg] Re: SRP HA dm_multipath testing and questions
>> 
>> Chieng Etta wrote:
>> > 
>> > Scott Weitzenkamp (sweitzen) wrote:
>> >> I've been testing SRP HA and dm_multipath with:
>> >> - RHEL4 U3 x86_64, Cisco FC Gateway, and Sun T4 RAID
>> >> - RHEL4 U3 x86_64, Cisco FC Gateway, and Sun 3510 RAID
>> >> - SLES10 x86_64, Cisco FC Gateway, and 3 JBODs
>> >>  
>> >> On RHEL4, I edited /etc/multipath.conf, ran "chkconfig 
>> multipathd on", 
>> >> then rebooted.  On SLES 10, I ran "chkconfig 
>> boot.multipath on" and 
>> >> "chkconfig multipathd on", then rebooted.  Ishai, I don't 
>> seem to need 
>> >> 91-srp.rules, are you using the boot.multipath and 
>> multipathd scripts?
>> > 
>> > On RHEL4 you really do not need 91-srp.rules and it is not used (see
>> > /etc/init.d/openibd)
>> > On SLES10 I was sure that you need it. I checked it, and 
>> you are correct. I
>> > don't see how it does it, but it seems that when using 
>> boot.multipath there
>> > is no need for 91-srp.rules. I will check it more deeply and change
>> > documentation and openibd script accordingly. 
>> > 
>> > [EC] I just verified it on SLES10 x86_64.  The multipath 
>> worked fine by
>> > using boot.multipath without 91-srp.rules.
>> > 
>> In one of Novell's documents (SLES 10 Storage Administration 
>> Guide for EVMS - In section 5 Managing Multipath I/O for 
>> Devices 
>> http://www.novell.com/documentation/sles10/index.html?page=/do 
> cumentation/sles10/stor_evms/data/multipathing.html) it says in
> subsection 5.7 that after a new target > was discovered there is a need
> to actively execute multipath. 
>> (As I understand it from the document this is true even after 
>> boot.multipath is running) 
>> 
>> Experiments in my environment also indicates that after 
>> executing boot.multipath, SRP HA is working also without 
>> 91-srp.rules, but after reading this document I'm even more confused.
>> 
>> 
>> 
>> > Ishai, in the SRP release notes - section 6, srp_daemon a., 
>> the first line
>> > should be changed to '"srp_daemon -a -o" is equivalent to 
>> "ibsrpdm"'.
>> > 
>> > 
>> Thanks, However Scott already noticed that and I already 
>> fixed it. You will see it in the next documentation version.
>>