[ofa-general] Re: RFC: "mlx4" drivers for Mellanox ConnectX HCAs

[Michael S. Tsirkin]

> Quoting Roland Dreier <rdreier at cisco.com>:
> Subject: Re: RFC: "mlx4" drivers for Mellanox ConnectX HCAs
> 
>  > >   - Fix CQ locking on destroy QP; not completely trivial since I want
>  > >     to keep a somewhat clean division between mlx4_core and mlx4_ib
>  > 
>  > It seems core could just export a "cleanup" functions then, and
>  > mlx4_ib would call that under the appropriate lock?
> 
> Yes, I don't think it's a huge issue, I just need to figure out a way
> to implement it that I like.  The core already exports __mlx4_qp_lookup()
> so something like __mlx4_qp_remove() would make sense.
> 
>  > >   - clean stale CQEs on destroy QP or modify QP to RESET
>  > 
>  > BTW, I went over that code in mthca and it seems that it does
>  > not handle CQ resize correctly. Right?
> 
> I don't know of a problem -- or do you mean that it fails if a CQ
> resize is in progress as a QP destroy/reset occurs?

I haven't seen this in practice but it seems so from reading the code.

The code that does CQ cleanup only looks in the primary buffer.
So if resize is in progress, it seems CQEs for our QP could be left in the
resize buffer, and when we do lookup by QPN it will fail or get
the wrong QP.

>  > >   - inline send support
>  > 
>  > Inline send from userspace, or from kernel as well?
>  > If from kernel - note that we never had inline
>  > in kernel for older HW, so ULPs don't use it.
>  > So I guess this is a low priority feature?
> 
> I guess userspace is a higher priority.  We could ignore it for the
> kernel for the moment.
> 
>  > > And of course we still need to get write combining support in the core
>  > > kernel to make blueflame work well.
>  > 
>  > You haven't gootten anhy feedback from your last request for
>  > comments on this, did you?
> 
> No, but I haven't pushed very hard on it yet.  I think this is a
> 2.6.23 thing at this point.

Right. My approach would be to implement for 2.6.22 blueflame + inline,
without write combining. WC can then be enabled by either
userspace script playing with MTRRs, or user patching in PAT
support.

-- 
MST