[ofa-general] Re: [PATCH draft, untested] ehca srq emulation (for IPoIB CM)

[Michael S. Tsirkin]

> Quoting Roland Dreier <rdreier at cisco.com>:
> Subject: Re: [ofa-general] Re: [PATCH draft,?untested] ehca srq emulation (for IPoIB CM)
> 
>  > > Note this is not a full emulation, just close enough to make IPoIB CM work.
> 
>  > If the emulation is only enough for IPoIB, then I think it belongs in
>  > IPoIB, and not in every HCA driver.

"every HCA driver" is an exagerration:
1. ehca is the only one that does not support SRQ in hardware
2. emulation (and ipoib nosrq patches, too) work by assuming only a
   small number of connections and a huge amount of memory.
   This is true for systems where ehca is used but not in the general case

> I was thinking the same thing.  Otherwise you're just setting a booby
> trap for someone who tries to use SRQ for something else.

The emulation is quite close IMO - most likely it will just work,
but if not, we can just document the limitations.

In case a ULP wants to avoid using the emulation, we could have a "SRQ is
emulated bit" to distinguish between these.

> However it may be a good approach to put an abstraction layer in IPoIB
> so that the CM code can use an SRQ-like interface to both HCAs that
> support SRQ and HCAs that don't.

2 issues with this:

1. I think other ULPs can benefit from this emulation too.
2. The emulation does need help from hardware (e.g. I use a qp token
   in CQE for QP lookups and SRQ detection).
   Implementing it on top of exiting verbs can be done
   only if verbs interface is extended.



-- 
MST