[ofiwg] mapping adapter memory

[Tsai-yang Jea]

I can see both of these options would be set all the time.  Users are
greedy. They probably want both good latency and good bandwidth.

Unless, there is a clear documentation that describes the limitations and
drawbacks by enabling these options (however, the limitation and drawbacks
is provide specific). Then, users may choose carefully.

In general, I think users will set both options since they are ignored if
hardware does not support them. Who doesn't want an EP that has both good
bandwidth and latency? If both of the options are always set, or set most
of the time, it is the same as not having the options.

Regards,

Alan



From:	"Reese Faucette (rfaucett)" <rfaucett at cisco.com>
To:	"Hefty, Sean" <sean.hefty at intel.com>,
            "ofiwg at lists.openfabrics.org" <ofiwg at lists.openfabrics.org>
Date:	09/24/2014 11:03 PM
Subject:	Re: [ofiwg] mapping adapter memory
Sent by:	ofiwg-bounces at lists.openfabrics.org



> Conceptually, I can see where not all endpoints may have dedicated
> hardware behind them and may have to share resources with other
> endpoints, and potentially other processes.  Even adapters that can
> dedicate hardware resources to every endpoint may not perform well as a
> result of caching limitations on the HCA.  This could require an app to
share
> resources (e.g. a kernel allocated QP) for specific communication
channels.
>
> Maybe a provider can expose some attributes on the 'optimal' use of any
> the underlying hardware, so that an application or job scheduler doesn't
> oversubscribe the hardware.  Reporting maximum values doesn't do that,
> since apps often allocate the max values expecting that there won't be
any
> performance loss for doing so.

How about something like this:
There are hints the app can specify when creating an EP, such as
"OPTIMIZE_FOR_LATENCY" or "OPTIMIZE_FOR_BW".  If the hardware has nothing
special to do for those hints, they are ignored.  If there are some special
hardware resources that can do one or the other, the provider will make a
best-effort to match them with their respective hints.  Once these special
resources are exhausted, "you get what you get".  So, if hardware supports
N "low latency" QPs, and the app requests N+1 OPTIMIZE_FOR_LATENCY QPs, the
N+1th QP will just be a little slower than the others.  Moral: ask for the
more important ones first.

That seems not-to-invasive, and is effective for my needs.
-reese
_______________________________________________
ofiwg mailing list
ofiwg at lists.openfabrics.org
http://lists.openfabrics.org/mailman/listinfo/ofiwg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofiwg/attachments/20140926/116c8582/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/ofiwg/attachments/20140926/116c8582/attachment.gif>