[ofa-general] Re: [PATCH draft, untested] ehca srq emulation (for IPoIB CM)

[Michael S. Tsirkin]

> Quoting Sean Hefty <mshefty at ichips.intel.com>:
> Subject: Re: [ofa-general] Re: [PATCH draft,?untested] ehca srq emulation (for IPoIB CM)
> 
> >Basically, I think that because of lack of SW level flow control,
> >generally IPoIB CM without SRQ does not make sense because of
> >the scalabilty problems.
> 
> Most clusters are only 16-32 nodes.
>
> If IPoIB CM without SRQ can support 
> this number of systems and outperforms IPoIB UD mode, then I do believe 
> that it makes sense.

Note that e.g. with mthca regular QP has lower overhead than SRQ
(less locking, etc). So if your assumption on the number of nodes in IB
clusters is generally correct, we need a generic layer that will start with
regular QPs for a small number of connections, then switch to
SRQ as the number of connections grows (and to datagram mode
if SRQ is not available).

> IPoIB CM support, with or without SRQ, is less 
> scalable than IPoIB UD mode,

I believe this is incorrect: datagram mode has AH per destination,
connected mode has a QP per destination, so with SRQ, I see no
inherent lack of scalability with connected as compared to datagram mode.

> but it was still added because it provided 
> a benefit under most conditions.

-- 
MST