[ofa-general] Re: Incorrect max_sge reported in mthca device query

[Tom Tucker]

On Mon, 2007-04-02 at 09:08 +0300, Michael S. Tsirkin wrote:
> > On Sun, 2007-04-01 at 09:43 +0300, Michael S. Tsirkin wrote:
[...snip...]
> I think that if we extend the API, we need to design it carefully
> to cover as many use cases as possible.
> Tom, could you explain what are you trying to do?
> Why does your application need as many SGEs as possible?
> 
Mike:

The application is NFS-RDMA. NFS keeps it's data as non-contiguous
arrays of pages. So the motivation is that having a larger SGL allows
you to support larger data transfers with a single operation. 

The challenge with the current query/request method is that as we've
discussed the advertised max may not work. What makes the adjust/retry
unworkable is that you don't know which of the advertised maxes caused
the request to fail. So when you retry, which qp_attr do you adjust? The
send sge? The recv sge? The qp depth?

So what I'm proposing, and I think is similar if not identical to what
other folks have talked about is having an interface that treats the
qp_attr values as requested-sizes that can be adjusted by the provider.
So for example, if I ask for a send_sge of 30, but you can only do 28,
you give me 28 and adjust the qp_attr structure so that I know what I
got. This would allow me to perform a predictable sequence of 1. query,
2. request, 3. adjust in my code.

BTW, I think it needs to be new provider method to be done efficiently.
Also, what's a good name, ib_request_qp? 

Thanks,
Tom

> Also - what about out of resources cases described above?
> Would you expect the verbs API to retry the request for you?
>