[ofa-general] MLX HCA: CQ request notification for multiple completions not implemented?

[Dotan Barak]

Yicheng Jia wrote:
>
> >> My concern is the performance. The overhead of software poll_cq loop
> >> is quite significant if there are multiple pieces of small amount of
> >> data to be transferred on both sender/receiver sides. For instance, on
> >> the sender, the data I have are 1k, 1k, 2k, 1k..., on the receiver
> >> side, the data size and blocks are the same, 1k, 1k, 2k, 1k.... Do you
> >> have a good solution for such kind of problem?
>
> >How many QPs do you use?
> >(and how outstanding WR from every QP?)
>
> Only one QP. Is it better to alloc multiple QPs and evenly distribute 
> WRs among those QPs?
It depends on what you are trying to do ...

You don't have to ask for completion for any SR that you post, this way 
you can do some optimization..


Dotan
>
> Best,
> Yicheng
>
>
>
> *Dotan Barak <dotanba at gmail.com>*
>
> 05/28/2008 12:24 PM
>
> 	
> To
> 	Yicheng Jia <YJia at tmriusa.com>
> cc
> 	general at lists.openfabrics.org
> Subject
> 	Re: [ofa-general] MLX HCA: CQ request notification for multiple 
> completions not implemented?
>
>
>
> 	
>
>
>
>
>
> Yicheng Jia wrote:
> >
> > Thanks for your reply. I'm using one CQ for all the WRs. Do you know
> > why there's no ARM-N support in MLX drivers?
> I don't know if i can speak in the name of Mellanox/MLX driver
> maintainers, but i think that the
> reason is lack of demand for this feature (but i can't be sure).
>
> > My concern is the performance. The overhead of software poll_cq loop
> > is quite significant if there are multiple pieces of small amount of
> > data to be transferred on both sender/receiver sides. For instance, on
> > the sender, the data I have are 1k, 1k, 2k, 1k..., on the receiver
> > side, the data size and blocks are the same, 1k, 1k, 2k, 1k.... Do you
> > have a good solution for such kind of problem?
> How many QPs do you use?
> (and how outstanding WR from every QP?)
>
> Dotan
> > Best,
> > Yicheng
> >
> >
> >
> > *Dotan Barak <dotanba at gmail.com>*
> >
> > 05/23/2008 01:27 PM
> >
> >                  
> > To
> >                  Yicheng Jia <YJia at tmriusa.com>
> > cc
> >                  general at lists.openfabrics.org
> > Subject
> >                  Re: [ofa-general] MLX HCA: CQ request notification 
> for multiple
> > completions not implemented?
> >
> >
> >
> >                  
> >
> >
> >
> >
> >
> > Hi.
> >
> > Yicheng Jia wrote:
> > >
> > > Hi Folks,
> > >
> > > I'm trying to use CQ Event notification for multiple completions
> > > (ARM_N) according to Mellanox Lx III user manual for scatter/gathering
> > > RDMA. However I couldn't find it in current MLX driver. It seems to me
> > > that only ARM_NEXT and ARM_SOLICIT are implemented. So if there are
> > > multiple work requests, I have to use "poll_cq" to synchronously wait
> > > until all the requests are done, is it correct? Is there a way to do
> > > asynchronous multiple send by subscribing for a ARM_N event?
> > You are right: the low level drivers of Mellanox devices doesn't support
> > ARM-N
> > (This feature is supported by the devices, but it wasn't implemented in
> > the low level drivers).
> >
> > You are right, in order to read all of the completions you need to use
> > poll_cq.
> >
> > By the way: Do you have you have to create a completion for any WR?
> > (if you are using one QP, this will maybe solve your problem).
> >
> > Dotan
> >
> > 
> _____________________________________________________________________________
> > Scanned by IBM Email Security Management Services powered by
> > MessageLabs. For more information please visit http://www.ers.ibm.com
> > 
> _____________________________________________________________________________
> > < <http://www.ers.ibm.com/>http://www.ers.ibm.com/>
> >
> > 
> _____________________________________________________________________________
> > Scanned by IBM Email Security Management Services powered by
> > MessageLabs. For more information please visit http://www.ers.ibm.com
> > 
> _____________________________________________________________________________
>
>
> _____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by 
> MessageLabs. For more information please visit 
> <http://www.ers.ibm.com/>http://www.ers.ibm.com
> _____________________________________________________________________________
> <http://www.ers.ibm.com/>
>
> _____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by 
> MessageLabs. For more information please visit http://www.ers.ibm.com
> _____________________________________________________________________________