[ofa-general] [RFC] XRC -- make receiving XRC QP independent of any one user process

[Tang, Changqing]

> -----Original Message-----
> From: Jack Morgenstein [mailto:jackm at dev.mellanox.co.il]
> Sent: Friday, December 21, 2007 2:51 PM
> To: Tang, Changqing
> Cc: pasha at dev.mellanox.co.il; general at lists.openfabrics.org
> Subject: Re: [ofa-general] [RFC] XRC -- make receiving XRC QP
> independent of any one user process
>
> On Friday 21 December 2007 20:22, Tang, Changqing wrote:
> > What we do for heart-beat is using zero-byte rdma_write,
> the message
> > goes to the peer QP only, there is no need to post anything
> on remote side, no need for pinned memory.
> >
> I'll look into this solution on Sunday (I've not used 0-byte
> rdma_reads myself yet).
> (Question -- does the 0-byte rdma-read need to access a valid
> address (i.e., region) on the remote side, even if it is
> zero-byte? or are the remote address and rkey fields "don't
> care" in the post_send work request in this case?)

We need to ask Roland to confirm this.


>
> If you can, please send me a coding example at the userspace
> ibv (verbs) layer.
> It will save time.

I did not use zero byte rdma_read, I only use zero byte rdma_write.

Here is our code:

                sr.next = NULL;
                sr.wr_id = (uint64_t)(AULONG)rdmahdr;

                sr.sg_list = &ssg;
                sr.num_sge = 0;
                sr.opcode = IBV_WR_RDMA_WRITE;
                sr.send_flags = IBV_SEND_INLINE|IBV_SEND_SIGNALED;

                err = ibv_post_send(ibvproc->connection[i].qp_hndl, &sr, &bad_sr);
                if (err != 0) {
                        hpmp_printf("ibv_post_send() failed");
                        return (-1);
                }

Note, ssg is not initialized (Maybe we can set sr.sg_list = NULL ?)


--CQ


> (jackm at dev.mellanox.co.il).
>
> Thanks!
>
> - Jack
>