[ofw] Need code examples

[Sean Hefty]

>I'm using RC QP connection but data in the buffer on receive side changes
>without any actions (ibv_post_recv) on receive side. I'm tried to use multiple
>buffers to send/recv data frames but on receive side only ONE buffer (first
>registered with ibv_reg_mr) works. I don't found any examples where used a pool
>of buffers for receive side (for sender it is). Would you help me in such
>things?

Just allocate an array of buffers, register the entire array, then post each
buffer separately:

// define these as whatever makes sense for your app
#define MAX_RECV_QUEUE_SIZE 100

struct recv_msg {
	char buffer[256];
};

...

	// allocate an array of buffers
	recvs = (struct recv_msg *)
		malloc(sizeof(struct recv_msg) * MAX_RECV_QUEUE_SIZE);

	// register the array
	recv_mr = ibv_reg_mr(pd, recvs,
		sizeof(struct recv_msg) * MAX_RECV_QUEUE_SIZE,
		IBV_ACCESS_LOCAL_WRITE);

...

	// post a bunch of receives - one at a time
	struct ibv_recv_wr recv_wr, *recv_failure;
	struct ibv_sge sge;
	int i, ret = 0;

	recv_wr.next = NULL;
	recv_wr.sg_list = &sge;
	recv_wr.num_sge = 1;
	recv_wr.wr_id = (uintptr_t) node;

	sge.length = sizeof(struct recv_msg);
	sge.lkey = recv_mr->lkey;

	for (i = 0; i < MAX_RECV_QUEUE_SIZE && !ret; i++) {
		sge.addr = (uintptr_t) &recvs[i];
		ret = ibv_post_recv(qp, &recv_wr, &recv_failure);
		if (ret) {
			printf("failed to post receives: 0x%x\n", ret);
			break;
		}
	}

- Sean