[ofa-general] Re: [PATCH draft, untested] ehca srq emulation (for IPoIB CM)

[Pradeep Satyanarayana]

Michael S. Tsirkin wrote:
>> Pushing the changes into the driver is a potential maintenance
>> nightmare. How does one keep changes across layers in sync?
> 
> We have different definitions of "across layers": in my code everything is kept
> inside ehca. I call it a maintenance nightmare when there's code in IPoIB that
> only ehca owners can test.
> 
>> That was the reason I strived to use common code in the NOSRQ case; at
>> least  as much as possible and all of it in IPoIB.
> 
> And you ended up with a bigger patch.
> 
>> In the emulation approach by apportioning off WRs across QPs, we will be
>> sacrificing performance by dropping packets or returning an RNR on a
>> really busy QP. As I see it, the alternative is to allocate a really big
>> SRQ, even when there are very few QPs and wasting a lot of the unused WRs.
> 
> As I said, there are obvious performance optimisatons to implement.
> We can later add code in IPoIB that, for very large SRQ size,
> will post WRs on demand. But at least that will be common code
> that everyone can test.
> 

Micheal,

That is exactly the point. I made some decisions that you may not
agree with entirely. Each solution has its benefits and draw
backs. I feel that for a "performance related patch", performance should
be one of the most important attributes. Some of the other issues are
secondary.

Here is a patch that is working and tested on multiple HCAs. If you
feel it needs to be embellished in certain ways, sure go ahead and
incorporate changes on top of my patch. After all this is open source
development.

At the same time I would have reservations about a patch that takes a
performance hit even though it may have other desirable attributes.

I have already incorporated several of your valuable suggestions into
this patch, even though I did not agree with all of them. I see no need
for us to take opposite sides on every issue, but rather we should work
more constructively.

This issue has dragged on for weeks without much forward progress. We
need to make some decisions and close out this issue at the earliest.

Pradeep