Selective signalling vs WQE size - was Re: [ofa-general] mthca max_sge value... ugh.

[Eli Cohen]

On Mon, 2008-05-19 at 15:27 -0700, Roland Dreier wrote:
> mlx4 has a feature that lets the driver post smaller WQEs to the send
> queue if not all s/g entries are used.  But the current implementation
> at least can only use this feature if selective signaling is off.
> 
> So it's a tradeoff -- more work completions or bigger data structures
> for the HCA to fetch.  In the NFS/RDMA case I would expect the selective
> signaling to be a win, but I guess the thing to do is try ConnectX
> without selective signaling and see which wins.
> 
Roland, I posted a few months ago a patch that optimizes post send for
selective signaling QPs. It must have slipped somehow because I did not
get any reply on it and since I did not know of anyone using selective
signaling I forgot about this too. The idea is that for selective
signaling QPs, before you stamp the WQE, you read the value of the DS
field which denotes the effective size of the descriptor as used in the
previous post, and stamp only that area, relying on the fact that the
rest of the descriptor is already stamped. Here is a link to the patch.
I don't know if it applies cleanly now but if we agree on the idea I
will generate it again against the current tree.

http://lists.openfabrics.org/pipermail/general/2008-January/045071.html