[Openib-windows] SRP tuning

[Andre Karobkoff]

This is excellent information.  We have seen and demonstrated the
performance boost with larger transfers in our Linux environment.  Would
anyone be able to answer my original question from several weeks ago as to
where the source is located?

Thanks in advance for any and all assistance.


Andre N. Karobkoff
Software Engineer/Analyst
CIPRICO Inc.
30851 Agoura Road, Suite 207
Agoura Hills, CA 91301
(818) 991-1188




 
-----Original Message-----
From: openib-windows-bounces at openib.org
[mailto:openib-windows-bounces at openib.org] On Behalf Of Fabian Tillier
Sent: Monday, October 02, 2006 12:34 PM
To: Yossi Leybovich
Cc: openib-windows at openib.org
Subject: Re: [Openib-windows] SRP tuning

Hi Yossi,

On 10/1/06, Yossi Leybovich <sleybo at mellanox.co.il> wrote:
>
> Fab
> One of the people in the list ask about tuning option to the SRP 
> driver
>
>
> "...Now in regards to performance - In the linux environment, on the 
> host/initiator there were numerous parameters that could be toyed with 
> to boost performance.  Specifically, there is a parameter called 
> max_xfer_sectors_per_io in a file called modprobe-openib.conf.  By 
> changing the value of this parameter, performance via srp to srpt and 
> finally to disks could go up and down dramatically..."
>
> Do you have nay idea of similar parameter in windows SRP driver ?
> I found  p_config->MaximumTransferLength       =
> SCSI_MAXIMUM_TRANSFER_SIZE;
> which is 1K*1K.
>
> Does this value can be change ? Do you think it will affect the 
> performance ?

We should probably leave this value uninitialized.  It is initialized by the
port driver to SP_UNINITIALIZED_VALUE to indicate unlimited maximum transfer
size.

The real limit to the size of transfers is the NumberOfPhysicalBreaks and
SrbExtensionSize.  The SRB extension is used to hold the SRP requests,
including the S/G list for direct and indirect I/O requests.
 The size of the SRB extension limits the number of S/G entries, and thus
the NumberOfPhysicalBreaks.

Right now, the SRB extension is large enough to hold a SRP IU of 340 bytes,
which is enough to send up to 17 S/G entries in the partial memory
descriptor list.  Note that this S/G list is referenced by the indirect
table memory descriptor, so even if the initiator to target IU size doesn't
support the 17 S/G entries to be put in the SRP command, the 17 S/G entry
limit still holds.

17 S/G entries is the maximum needed to transfer 64KB in a single I/O.

A target that defines a larger initiator to target IU will have larger SRB
extensions, and thus support a larger transfer size.

It would be intersting to see how the S/G limit affects bandwidth.
Increasing SRP_MAX_IU_SIZE will achieve this.  You won't run into the
SCSI_MAX_TRANSFER_SIZE limit until you have 257 S/G entries, which means a
SRP_MAX_IU_SIZE of 4180 bytes (from it's current 340 bytes).

- Fab

_______________________________________________
openib-windows mailing list
openib-windows at openib.org
http://openib.org/mailman/listinfo/openib-windows