[openib-general] Re: [PATCH] [SRP] support for it_iu length negotiation

[Fab Tillier]

> From: John Kingman [mailto:kingman at storagegear.com]
> Sent: Tuesday, November 01, 2005 7:26 AM
> 
> On Mon, 31 Oct 2005, Roland Dreier wrote:
> 
> >With that said I don't think I like this patch.  I don't think it's a
> >win to allocate 1 KB IUs when we'll almost never have gather/scatter
> >lists that big.  Even the 256 byte IUs that the current driver uses
> >seem on the borderline of being too big.
> >
> >Also, is it really a win to have the target fetch a large indirect
> >buffer list?  It seems like it would be better for performance to give
> >the SCSI layer a limit on the size of the gather/scatter list we
> >support so that our indirect buffer lists always fit in the IUs we send.
> 
> Without knowing what the optimal values should be, perhaps we should
> make some of these module parameters.

The Windows SRP initiator sizes the IU to be capable of performing a 64KB I/O
with all SGEs specified in the IU.  It takes 350 bytes to be able to put the
full SGL into an IDBD IU, assuming 4K pages.

An alternative is to always force a RDMA read of the SGL, and just go with the
minimum size IU.  I don't know how this would affect performance, though -
likely increased latencies.  In fact, in environments where each I/O buffer can
be registered (via regular or fast MR) on the fly, DDBD should be used and the
IU would be a constant 64 bytes.  This should yield the best performance.

- Fab