[openib-general] Re: Gen2 User verbs usage
Roland Dreier
roland at topspin.com
Fri Apr 15 13:19:27 PDT 2005
>>>>> "Sayantan" == Sayantan Sur <surs at cse.ohio-state.edu> writes:
Sayantan> Hi Roland, I have some questions regarding the usage of
Sayantan> the new Gen2 verbs.
Sayantan> 1. Polling CQ : I notice that this verb is little
Sayantan> different from VAPI_poll_cq, in the sense that it
Sayantan> accepts a parameter to poll for multiple completion
Sayantan> entries. So, if I have a statement like:
Sayantan> 497 ne = ibv_poll_cq(hca.cq, 1, &wc);
Sayantan> I want to poll for one completion. Does `ne' hold the
Sayantan> return status or number of elements actually pulled out
Sayantan> of CQ?
Sorry, this should be documented better in the userspace library. The
semantics are identical to the ib_poll_cq() function in the kernel:
* Poll a CQ for (possibly multiple) completions. If the return value
* is < 0, an error occurred. If the return value is >= 0, it is the
* number of completions returned. If the return value is
* non-negative and < num_entries, then the CQ was emptied.
Sayantan> 2. Posting RDMA write : Do these statements for
Sayantan> preparing a RDMA write IB descriptor make sense?
Sayantan> 472 sr_desc.send_flags = IBV_SEND_SIGNALED;
Sayantan> 473 sr_desc.opcode = IBV_WR_RDMA_WRITE;
Sayantan> 474 sr_desc.wr_id = 0;
Sayantan> 475 sr_desc.num_sge = 1;
Sayantan> 477 sr_desc.sg_list = &(sg_entry);
Sayantan> 479 sr_desc.wr.rdma.remote_addr = (uintptr_t) (rbuf.buf);
Sayantan> 480 sr_desc.wr.rdma.rkey = rbuf.rkey;
Sayantan> 483 sg_entry.addr = (uintptr_t) (lbuf.buf);
Sayantan> 484 sg_entry.length = len;
Sayantan> 485 sg_entry.lkey = lbuf.mr->lkey;
Sayantan> Essentially, I don't understand what the `send_flags'
Sayantan> field means.
Yes, this all makes sense. The send_flags field can hold any
combination (|'ed together) of the flags IBV_SEND_FENCE,
IBV_SEND_SIGNALED, IBV_SEND_SOLICITED and IBV_SEND_INLINE. FENCE
means that strict ordering will be enforced, as described in section
10.8 of the IB spec. IBV_SEND_SIGNALED means that a CQ entry will be
generated when the send is completed (this flag is ignored if the QP
is created with sq_sig_all != 0, since all sends will generate CQ
entries anyway). SOLICITED means that the solicited bit will be set
in the message so that the remote side will receive a solicited
completion event. INLINE means the verbs driver should try to copy
the data directly into the send work request to reduce latency.
Sayantan> On a side note, if you think that this sort of
Sayantan> discussion is useful in openib-general, please feel free
Sayantan> to Cc to that list.
Yes, I definitely think these questions should go through the mailing
list so that all the subscribers (plus any future archive searchers!)
can learn from the answers.
Thanks,
Roland
More information about the general
mailing list