[ofa-general] Re: RFC: "mlx4" drivers for Mellanox ConnectX HCAs

[Michael S. Tsirkin]

> Quoting Roland Dreier <rdreier at cisco.com>:
> Subject: Re: RFC: "mlx4" drivers for Mellanox ConnectX HCAs
> 
>  > a. How will I know whether or not I've got the most recent version with
>  >    all your changes -- does the commit ID change, for example?
> 
> Yes, commit ID will always be different, since it is in effect a SHA1
> hash of the full tree state (and a collision is unlikely, to say the
> least).
> 
>  > b. git fetch gives me problems, since the commits diverge -- my version
>  >    of the connectx branch is not a strict descendent of yours with respect to
>  >    the connectx changes, since the connectx commits themselves have changed.
>  >    (I of course do not fetch directly into my working branch, but into
>  >     an origin -- and this origin does not match the commits in your
>  >     connectx branch).
>  > 
>  > c. I think we need to start working with regular commits, so that 
>  >    each change can be documented, and git fetch/rebase can work smoothly.
> 
> Yes, I guess it is a pain now that we are collaborating actively.  OK,
> I'll just check into the branch normally, and we can collapse the
> patches down when I create a branch for Linus to pull.  I have one
> more rebase pending, and then I'll leave the branch alone.
> 
>  > Tziporet has requested that I implement the write-combining support.
>  > I'll be doing that first thing.
> 
> What is your plan for doing this?  I had the following vague idea:
> 
>  - Make pgprot_writecombine() a supported API for drivers that all
>    architectures should provide (it can be defined to be the same as
>    pgprot_noncached() for architectures where a write-combining
>    mapping doesn't make sense).
>  - Add an API ioremap_writecombine() that does the same thing for
>    kernel space.
>  - Add PAT-based implementations for x86-64 and i386 architectures.

I don't know what does Jack plan, but I would start with blueflame support
(without WC at first), then try to push just item 3, and use architecture
specific code.

It's not clear that write combining can be defined sufficiently
consistently across architrectures, or so I heard from
Gred and Andi, might be better to do it step by step.

And ioremap_writecombine isn't needed for mlx4 unless we have
kernel-level inline sends.

> Are you going to add inline send support for mlx4 (kernel and libmlx4) first?

I'm not yet sure kernel-level inlines make sense.
If they do, why don't we start by adding this support to mthca?

I think inline / blueflame support in libmlx4 should be a priority.

-- 
MST