[ofa-general] Re: IPOIB CM (NOSRQ) patch -memory footprint

[Pradeep Satyanarayana]

Michael S. Tsirkin wrote:
>>>> -The IPOIB CM (NOSRQ) patch is practically transparent to HCAs that
>>>> support SRQ like the Topspin HCA and, such HCAs should not be
>>>> impacted at all.
>>> I don't think it's that clean yet.
>>>
>>> Here's an idea: implement "fake SRQ" for ehca in software: make post recv on
>>> srq queue the WR, spread them evenly between QPs as they connect.  Once # of
>>> QPs goes above some limit, create QP command will fail. 

This already exists in the last couple of versions of the patch. We send
a REJ command when a predetermined threshold is crossed. What we have
been debating is what to do on the active side when the REJ command is
received.


  This would contain
>>> the mess nicely inside ehca (I think you'll want to add a flag that lets
>>> software figure out that SRQ is fake).
>>>
>>> We will still be left with the basic problem of what to do at the active side
>>> upon the reject, though.
>> As you indicate this will not solve the problem, so it is not an option.
> 
> Above, I have outlined how it can be done, so it certainly *is* an option.

In the previous mail I proposed a method to address both viewpoints:
a) let the active side return an error to the user level app and leave
the onus to the application b) switch to datagram mode when the QP
threshold is crossed.

There has been no response to that proposal.


> 
> In this thread, you basically keep saying that ehca will ever be the only HCA without SRQ
> support, so you can make a lot of assumptions about how IPoIB is used.
> 
> Fine, but if you follow this logic, it makes sense to hide the mess under the ehca
> provider interface.
> 
> 

Every time I address the issues you have raised previously, it appears
that something else crops up. I have said that I can provide a patch
that addresses both alternatives a) and b) above. Let us just stick to
that and limit our discussions and proceed to close the issues out and
not diverge any further.

Pradeep