[ofa-general] Re: Incorrect max_sge reported in mthca device query

[Tom Tucker]

On Thu, 2007-04-05 at 09:27 -0700, Sean Hefty wrote:
> >The challenge with the current query/request method is that as we've
> >discussed the advertised max may not work. What makes the adjust/retry
> >unworkable is that you don't know which of the advertised maxes caused
> >the request to fail. So when you retry, which qp_attr do you adjust? The
> >send sge? The recv sge? The qp depth?
> >
> >So what I'm proposing, and I think is similar if not identical to what
> >other folks have talked about is having an interface that treats the
> >qp_attr values as requested-sizes that can be adjusted by the provider.
> >So for example, if I ask for a send_sge of 30, but you can only do 28,
> >you give me 28 and adjust the qp_attr structure so that I know what I
> >got. This would allow me to perform a predictable sequence of 1. query,
> >2. request, 3. adjust in my code.
> 
> If the send sge/recv sge/qp depth/etc. aren't independent though, this pushes
> the problem and policy decision down to the provider.  I can't think of an easy
> solution to this.

Agreed. But practically I think they are. I think the SGE max is driven
off the max size of a WR and type of QP. This is true of the iWARP
adapters as well.  

But taking the bait...even if you didn't push it down to the provider,
how do you expose the inter-relationships to the consumer? An approach
in this vein is a "could_you_would_you/why_not" interface that would
return whether or not the specified qp_attr would work and if it didn't
some indication of which resource(s) caused the problem. The problems
there are a) the resource may be gone when you go back with what you
just had "approved", and b) you still have to fuss with multiple whacks
at it if you couldn't get what you asked for.

I think something simpler, although arguably not perfect is the way to
go.

Tom

> 
> - Sean