[ofa-general] [PATCH RFC v3 1/2] RDMA/Core: MEM_MGT_EXTENSIONS support

[Or Gerlitz]

Steve Wise wrote:
>>> Support for the IB BMME and iWARP equivalent memory extensions ... 
>>> Usage Model:
>>> - MR allocated with ib_alloc_mr()
>>> - Page lists allocated via ib_alloc_fast_reg_page_list().
>>> - MR made VALID & bound to a specific page list via 
>>> ib_post_send(IB_WR_FAST_REG_MR) - MR made INVALID via 
>>> ib_post_send(IB_WR_INVALIDATE_MR)
>> AFAIK, the idea was to let the ulp post --two-- work requests, where 
>> the first creates the mapping and the second sends this mapping to 
>> the remote side, such that the second does not start before the first 
>> completes (i.e a fence).
>>
>> Now, the above scheme means that the ulp knows the value of the 
>> rkey/stag at the time of posting these two work requests (since it 
>> has to encode it in the second one), so something has to be clarified 
>> re the rkey/stag here, do they change each time this MR is used? how 
>> many bits can be changed, etc.
>
> The ULP knows the rkey/stag because its returned up front in the 
> ib_alloc_fast_reg_mr().  And it doesn't change (ignoring the key issue 
> which we haven't exposed yet to the ULP).  The same rkey/stag can be 
> used for multiple mappings.  It can be made invalid at any point in 
> time via the IB_WR_INVALIDATE_MR so the fact that you're leaving the 
> same rkey/stag advertised is not a risk.
I understand that this (same rkey/stag used for all mapping produced for 
a specific mr) is what you are proposing, I still think there's a chance 
that by the spec and (not less important!) by existing HW support, its 
possible to have a different rkey/stag per mapping done on an mr, for 
example the IB spec uses a "consumer owned key portion of the L_Key" 
notation which makes me think there should be a way to have different 
rkey per mapping, Roland? Dror?
> 10.7.2.6 FAST REGISTER PHYSICAL MR
> The Fast Register Physical MR Operation is allowed on Non-Shared 
> Physical Memory Regions that were created with a Consumer owned key 
> portion of the L_Key, and any associated R_Key
Or