[openib-general] RE: [RFC] DAT 2.0 immediate data proposal

[Davis, Arlin R]

Arkady,

 

Response inline...

 

________________________________

From: Kanevsky, Arkady [mailto:Arkady.Kanevsky at netapp.com] 
Sent: Tuesday, January 17, 2006 7:16 AM
To: Davis, Arlin R; Lentini, James
Cc: dat-discussions at yahoogroups.com; openib-general at openib.org
Subject: RE: [RFC] DAT 2.0 immediate data proposal

 

Arlin,

a few things need to be addressed.

 

1. correlation with local and remote invalidate

This potentially effects both DAT_DTOs and post operations

 

 

How does this differ from normal sends or writes?

 

 

2. Need a precise defintion for CONFIRM_FLAG definition in a transport
independent fashion.

What guarantees DAT Provider "provides" on successful local completion?

Remote end guarantee?

 

My understanding what you are trying to do is create 2 models one IB and
one for iWARP.

So for IB Consumers will use CONFIRM_FLAG and for iWARP IMMED_FLAG.

Provider will indicate in Provider_attr which model it supports.

 

The issue I have with it is that I do not see a model that Consumer can
use to create

a transport independent code.

It looks like Immed_flag can be made transport independent. But with
"sender" specifying

the behavior a protocol extension is needed for IB. IB will always
deliver Immediate data

in the header not a payload and remote Provider can control how it is
delivered to a Consumer.

But this means that there is no need for DTO_flags for Send side.
Instead it can be

used for Recv side or controlled purely by Provider.

 

Maybe we need to just go back to one model and always deliver via the
event? With the post_recv_immed requirements, other transports have a
mechanism to emulate and create the necessary resources on the recv side
to place idata and copy to event when operation is completed. Would this
work for iWARP?

 

Two different models for receiving idata should be avoided if at all
possible.

 

3. Need to define error behavior. for new operations, async errors, EP
behavior.

 

I will work on updating the draft. post_send_immed will look much like
post_send and post_rdma_write_immed will look a lot like post_rdma_write
with some additional errors based on the post receive buffer
requirement.  

 

4. Need to define DAT_Provider attributes for immediate data and
dto_flags behavior

 

 

5. Does Solicited_wait completion_flag value now applicable for
RDMA_write for immediate data?

 

yes, applicable to  send, send_immed, and write_immed

 

6. Is dto_completion_data xfer_length include immediate_data size or
not?

 

no

 

7. what memory privilages needed for a recv buffer for immediate data?

 

Based on the operation... write_immed would require write privileges and
send_immed would require recv privileges.

 

8. SRQ interaction?

 

Good question. all post_recv_immed or all post_recv?  

 

9. What happens of buffer for recv operation NOT recv_immed is matched
for incomming recv/rdma_write op?

 

The rules should be:

Can receive a send, send_immed, or write_immed with recv_immed. 

Cannot receive send_immed or write_immed on a recv.

 

However, I am not sure how you would enforce this on IB (DTO error on
the receiving side?) since the idata is delivered via CQ and does not
require a special receive post descriptor.      

 

10. Change dat_ep_post_write_immed to dat_ep_post_rdma_write_immed to be
consistent with current

terminology.

 

Ok

 

11. Need to cleanup operation description to make it clear that
Send|RDMA_write and immediate data part

is a single atomic operation. The current "followed by" language is
misleading.

Make it explicit that there is a single local DTO completion and single
remote DTO completion.

 

Ok, I will clean that up

 

12. Is your intension that post_recv_immed can ONLY except immediate
data and is not

capable to recv any message?

 

No, the intention is to extend the post_recv to handle 32bit idata which
may arrive with or without other send or rdma_write data.

 

Does it make more sense to add a dto_flags to the existing post_recv?

 

13. size should be num_segments for dat_ep_post_recv_immed()

 

ok

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20060123/60c10b5f/attachment.html>