***SPAM*** Re: [ofa-general] non zero lkey in send(), write() with num_sge > 1?

[Dotan Barak]

Bill N wrote:
>>> Can stack pass num_sge > 1, and lkey !=0 as part of
>>>       
>> sg_list[] elements, in post_send() call?
>>     
>>>   
>>>       
>> What are you trying to achieve?
>>     
> [Bill]
> I just wanted to confirm, that even when Stag !=0,
> (a) there can be multiple SGEs in the list with different lkey and TO.
> And
> (b) HCAs have to validate each of the SGE entry against the lkey.
>
> Want to ensure that 
> - As RDMA ULP I can invoke post_send() with multiple lkeys and utilize the allocated MRs, HCAs are designed to handle that.
>
> Any example ULP we are aware of that does this?
>
> Regards,
> Bill
>   
If we are talking about the following scenario:
For example: num_sge = 3.

sg_list[0].lkey=A
sg_list[1].lkey=B
sg_list[2].lkey=C


so, here is the answer:

I checked the ULPs code which are part of the Linux kernel and I noticed 
that there isn't any ULP that uses several
SGEs from different memory regions:
Most of the ULPs uses only one SGE, and those who use more than one, use 
the same lkey.

 From my experience, I can tell you that the OFED stack support this feature
(and many HCAs support it too).

If you know otherwise, there is a bug somewhere..

Dotan