[ofa-general] post_recv question

[Steve Wise]

Hey all:

I have a question regarding exactly _when_ a posted recv buffer is 
available for the HW to use:

Consider that the post_recv methods usually just program a hw-specific 
WR in the RQ, then ring a doorbell, then return.  There is a delta 
period between when the app returns from the post_recv call and when the 
HW actually DMA's the WR and programs up the HW to enable that buffer. 
(I'm assumming a specific HW design here, but I _think_ most HW behaves 
this way?).

If this is all true, then from the apps point of view, the buffer isn't 
really available when it returns from post_recv.  This can lead to 
conditions where the app advertises that recv buffer to the peer via 
some out of band channel, and the peer posts a SEND which arrives 
_before_ the HW has actually setup the RECV buffer.

Granted, this hole is small, but does it exist for nes, mthca, ehca, and 
ipath libs/drivers?  Or do they _not_ have this issue?

Does the IBTA spec discuss this at all?  Most importantly, does the IBTA 
spec and/or the iWARP verbs spec _mandate_ that the buffer is actually 
available when the post_recv() method returns (I didn't find it in the 
iWARP spec)?  If such a mandate exists, then it would force post_recv() 
methods to stall and/or somehow know when the HW has completed setting 
up the recv buffer.  This would kill performance IMO and I think no such 
mandate exists, but I wanted to know what others think.

Maybe this isn't an issue with mthca/ehca/ipath/nes?


Thanks,

Steve.