[Openib-windows] SRP tuning

[Yossi Leybovich]

> -----Original Message-----
> From: openib-windows-bounces at openib.org 
> [mailto:openib-windows-bounces at openib.org] On Behalf Of Fabian Tillier
> Sent: Monday, October 02, 2006 9:34 PM
> To: Yossi Leybovich
> Cc: openib-windows at openib.org
> Subject: Re: [Openib-windows] SRP tuning
> 
> Hi Yossi,
> 
> On 10/1/06, Yossi Leybovich <sleybo at mellanox.co.il> wrote:
> >
> > Fab
> > One of the people in the list ask about tuning option to the SRP 
> > driver
> >
> >
> > "...Now in regards to performance - In the linux 
> environment, on the 
> > host/initiator there were numerous parameters that could be 
> toyed with 
> > to boost performance.  Specifically, there is a parameter called 
> > max_xfer_sectors_per_io in a file called modprobe-openib.conf.  By 
> > changing the value of this parameter, performance via srp 
> to srpt and 
> > finally to disks could go up and down dramatically..."
> >
> > Do you have nay idea of similar parameter in windows SRP driver ?
> > I found  p_config->MaximumTransferLength       =
> > SCSI_MAXIMUM_TRANSFER_SIZE;
> > which is 1K*1K.
> >
> > Does this value can be change ? Do you think it will affect the 
> > performance ?
> 
> We should probably leave this value uninitialized.  It is 
> initialized by the port driver to SP_UNINITIALIZED_VALUE to 
> indicate unlimited maximum transfer size.
> 
> The real limit to the size of transfers is the 
> NumberOfPhysicalBreaks and SrbExtensionSize.  The SRB 
> extension is used to hold the SRP requests, including the S/G 
> list for direct and indirect I/O requests.
>  The size of the SRB extension limits the number of S/G 
> entries, and thus the NumberOfPhysicalBreaks.
> 
> Right now, the SRB extension is large enough to hold a SRP IU 
> of 340 bytes, which is enough to send up to 17 S/G entries in 
> the partial memory descriptor list.  Note that this S/G list 
> is referenced by the indirect table memory descriptor, so 
> even if the initiator to target IU size doesn't support the 
> 17 S/G entries to be put in the SRP command, the 17 S/G entry 
> limit still holds.
> 
> 17 S/G entries is the maximum needed to transfer 64KB in a single I/O.
> 
> A target that defines a larger initiator to target IU will 
> have larger SRB extensions, and thus support a larger transfer size.
> 
> It would be intersting to see how the S/G limit affects bandwidth.
> Increasing SRP_MAX_IU_SIZE will achieve this.  You won't run 
> into the SCSI_MAX_TRANSFER_SIZE limit until you have 257 S/G 
> entries, which means a SRP_MAX_IU_SIZE of 4180 bytes (from 
> it's current 340 bytes).

I set:

SCSI_MAXIMUM_TRANSFER_SIZE = SP_UNINITIALIZED_VALUE.
SrbExtensionSize = 3552

I played with the NumberOfPhysicalBreaks:
The StorPort driver set the of NumberOfPhysicalBreaksvalue to 17 ( I don't
know why , Can I configure?)
But even if I return 16/17 as my supported NumberOfPhysicalBreaks it keep
post IOs with 8 s/g list size (32K)
[when I set it to 4 ,storPort send IOs with 4 s/g , So the value did effect
when I reduce it]

Any idea why ?




> 
> - Fab
> 
> _______________________________________________
> openib-windows mailing list
> openib-windows at openib.org
> http://openib.org/mailman/listinfo/openib-windows
>