[openib-general] [PATCH 3/7] IB/ipath - performance improvements via mmap of queues

[Ralph Campbell]

On Wed, 2006-08-23 at 14:50 -0700, Roland Dreier wrote:
> I applied this, but I'm wondering if this:
> 
>  > +int ipath_resize_cq(struct ibv_cq *ibcq, int cqe)
>  >  {
>  > +	struct ipath_cq		       *cq = to_icq(ibcq);
>  > +	struct ibv_resize_cq		cmd;
>  > +	struct ipath_resize_cq_resp	resp;
>  > +	size_t				size;
>  > +	int				ret;
>  > +
>  > +	pthread_spin_lock(&cq->lock);
>  > +	/* Save the old size so we can unmmap the queue. */
>  > +	size = sizeof(struct ipath_cq_wc) +
>  > +		(sizeof(struct ipath_wc) * cq->ibv_cq.cqe);
>  > +	ret = ibv_cmd_resize_cq(ibcq, cqe, &cmd, sizeof cmd,
>  > +				&resp.ibv_resp, sizeof resp);
>  > +	if (ret) {
>  > +		pthread_spin_unlock(&cq->lock);
>  > +		return ret;
>  > +	}
>  > +	(void) munmap(cq->queue, size);
>  > +	size = sizeof(struct ipath_cq_wc) +
>  > +		(sizeof(struct ipath_wc) * cq->ibv_cq.cqe);
>  > +	cq->queue = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED,
>  > +			 ibcq->context->cmd_fd, resp.offset);
>  > +	ret = errno;
>  > +	pthread_spin_unlock(&cq->lock);
>  > +	if ((void *) cq->queue == MAP_FAILED)
>  > +		return ret;
>  > +	return 0;
>  > +}
> 
> works against an old kernel driver.  It seems you do have this:
> 
>  > +	if (dev->abi_version == 1) {
>  > +		context->ibv_ctx.ops.poll_cq       = ibv_cmd_poll_cq;
>  > +		context->ibv_ctx.ops.post_srq_recv = ibv_cmd_post_srq_recv;
>  > +		context->ibv_ctx.ops.post_recv     = ibv_cmd_post_recv;
>  > +	}
> 
> so I guess you're just ignoring the failure of mmap() or something?
> 
>  - R.

Not quite. If the kernel driver is old, libipathverbs is using the
old functions which make system calls instead of doing the newer
mmap stuff. libipathverbs doesn't need to attempt calling mmap()
if it knows the driver doesn't support it.