[openib-general] Re: calculate inline data size

[Or Gerlitz]

How about this (trivial) approach: given a requestred SG size for the send queue of a created QP, the user gets the max inline
data she can do sends/rdma-writes with this QP. If the user want to have the max inline size supported, she creates the QP
with the max SG size supported, where the latter is an attribute retrieved by the query hca verb. 
 
Typically, inline sends/rdma-writes are cool for latency benchmarks/apps as eliminating the pci transaction saves ~1us 
on PCI-X and ~0.5us on PCI-EXP. Now, latency benchmarks typically test with zero payload size, and generally, latency 
is an important measure for "short" messeges, "short" should be something between 0-256 bytes. With the VAPI gen1 driver
I recall that SG size of 16 gives max inline size of 400 something. If with mthca its more or less the same, i think we are done here.
 
Or.
________________________________

From: openib-general-bounces at openib.org on behalf of Sean Hefty
Sent: Fri 7/8/2005 9:38 PM
To: Gleb Natapov
Cc: openib-general at openib.org
Subject: RE: [openib-general] Re: calculate inline data size



>> By "current behavior" I meant that create_qp should fail if an inline size is
>> specified that is too large.
>>
>But how should I know what maximum size that I can use? By trial and
>error?

I don't have a good answer to that yet.  I think that it should come from either
the device properties or some sort of query routine.  But I'm unclear on the
details.

One thought that I had was to rename "inline" sends to "non-registered" sends
and support any arbitrary size.  The device driver could then optimize transfers
based on any method that it chose, whether performing an inline transfer or
copying into pre-allocated buffers.  This seems to push too much policy into the
device driver though.  (I didn't say it was a good thought...)

I'm not sure about tying the inline size together with the number of sge's,
since that implies a specific implementation.  But then I can't think of a
reasonable alternative implementation...

Maybe the best solution is to keep what's there, but provide documentation on
how to calculate the actual amount of inline data that can be sent with any
given work request.  Or, provide a query (possibly an inline function or macro)
that takes a formatted work request as input and returns the maximum inline size
for that request.

- Sean

_______________________________________________
openib-general mailing list
openib-general at openib.org
http://openib.org/mailman/listinfo/openib-general

To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general