<html>
<body>
<font size=3>At 04:30 PM 3/20/2006, Talpey, Thomas wrote:<br>
<blockquote type=cite class=cite cite="">At 06:00 PM 3/20/2006, Sean
Hefty wrote:<br>
>Can you provide more details on this statement? When are you
fencing the send <br>
>queue when using memory windows?<br><br>
Infiniband 101, and VI before it. Memory windows fence later
operations<br>
on the send queue until the bind completes. It's a misguided attempt
to<br>
make upper layers' job "easier" because they can post a bind
and then<br>
immediately post a send carrying the rkey. In reality, it introduces
bubbles<br>
in the send pipeline and reduces op rates
dramatically.</font></blockquote><br>
The requirement / semantics were derived from the ULP being used to
construct the technology. The combination of a bind-n-send
operation was to reduce the software interactions with the device by
consolidating this into a combo operation. I do not follow your
logic that this creates a bubble in the send pipeline as there were also
ordering and correctness issues w.r.t. subsequent operations to the
send. The bind-n-send is a single operation and its fence semantics
were required to allow the bind to complete before informing the remote
of the subsequent information in order to avoid race conditions.
<br><br>
<blockquote type=cite class=cite cite=""><font size=3>I argued against
them in iWARP verbs, and lost. If Linux could introduce<br>
a way to make the fencing behavior optional, I would lead the
parade.<br>
I fear most hardware is implemented otherwise.</font></blockquote><br>
Hardware generally implements operations in the order they are posted to
a given QP, i.e. it is a serial execution flow that allows pipelined
operations to be posted and executed by the hardware. Scaling is
achieved by executing across a set of QP and thus a set of
resources. The ordering domain requirements are kept simple to
allow low-cost hardware implementations. This does not preclude
software from executing across a set of QP in any order that it
desires. <br><br>
<blockquote type=cite class=cite cite=""><font size=3>Yes, I know about
binding on a separate queue. That doesn't work, because windows are
semantically not fungible (for security
reasons).</font></blockquote><br>
You could always simply allow a region to be accessible across multiple
operations but then again storage argued that it must only be accessible
for a single op thus things like FMR, bind-n-send, etc. were all
created. To say that storage was not listened to or their needs
were not met or balanced against what is practical to implement in either
the creation of IB or iWARP is simply incorrect. <br><br>
Mike<br><br>
<br>
<blockquote type=cite class=cite cite=""><font size=3>Tom.<br><br>
_______________________________________________<br>
openib-general mailing list<br>
openib-general@openib.org<br>
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a><br><br>
To unsubscribe, please visit
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a>
</font></blockquote></body>
</html>