[ofa-general] [PATCH RFC v3 1/2] RDMA/Core:MEM_MGT_EXTENSIONS support

[Felix Marti]

> -----Original Message-----
> From: general-bounces at lists.openfabrics.org [mailto:general-
> bounces at lists.openfabrics.org] On Behalf Of Talpey, Thomas
> Sent: Tuesday, May 27, 2008 9:39 AM
> To: Tom Tucker
> Cc: Roland Dreier; general at lists.openfabrics.org
> Subject: Re: [ofa-general] [PATCH RFC v3 1/2]
> RDMA/Core:MEM_MGT_EXTENSIONS support
> 
> At 11:33 AM 5/27/2008, Tom Tucker wrote:
> >So I think from an NFSRDMA coding perspective it's a wash...
> 
> Just to be clear, you're talking about the NFS/RDMA server. However,
> it's
> pretty much a wash on the client, for different reasons.
> 
> >When posting the WR, We check the fastreg capabilities bit +
transport
> >type bit:
> >If fastreg is true -->
> >       Post FastReg
> >       If iWARP (or with a cap bit read-with-inv-flag)
> >               post rdma read w/ invalidate
> 
> >... For iWARP's case, this means rdma-read-w-inv,
> >plus rdma-send-w-inv, etc...
> 
> 
> Maybe I'm confused, but I don't understand this. iWARP RDMA Read
> requests
> don't support remote invalidate. At least, the table in RFC5040 (p.22)
> doesn't:
> 
> 
> 
>
-------+-----------+-------+------+-------+-----------+-------------
> -
>    RDMA   | Message   | Tagged| STag | Queue | Invalidate| Message
>    Message| Type      | Flag  | and  | Number| STag      | Length
>    OpCode |           |       | TO   |       |           |
Communicated
>           |           |       |      |       |           | between DDP
>           |           |       |      |       |           | and RDMAP
>
-------+-----------+-------+------+-------+-----------+-------------
> -
>    0000b  | RDMA Write| 1     | Valid| N/A   | N/A       | Yes
>           |           |       |      |       |           |
>
-------+-----------+-------+------+-------+-----------+-------------
> -
>    0001b  | RDMA Read | 0     | N/A  | 1     | N/A       | Yes
>           | Request   |       |      |       |           |
>
-------+-----------+-------+------+-------+-----------+-------------
> -
>    0010b  | RDMA Read | 1     | Valid| N/A   | N/A       | Yes
>           | Response  |       |      |       |           |
>
-------+-----------+-------+------+-------+-----------+-------------
> -
>    0011b  | Send      | 0     | N/A  | 0     | N/A       | Yes
>           |           |       |      |       |           |
>
-------+-----------+-------+------+-------+-----------+-------------
> -
>    0100b  | Send with | 0     | N/A  | 0     | Valid     | Yes
>           | Invalidate|       |      |       |           |
>
-------+-----------+-------+------+-------+-----------+-------------
> -
>    0101b  | Send with | 0     | N/A  | 0     | N/A       | Yes
>           | SE        |       |      |       |           |
>
-------+-----------+-------+------+-------+-----------+-------------
> -
>    0110b  | Send with | 0     | N/A  | 0     | Valid     | Yes
>           | SE and    |       |      |       |           |
>           | Invalidate|       |      |       |           |
>
-------+-----------+-------+------+-------+-----------+-------------
> -
>    0111b  | Terminate | 0     | N/A  | 2     | N/A       | Yes
>           |           |       |      |       |           |
>
-------+-----------+-------+------+-------+-----------+-------------
> -
>    1000b  |           |
>    to     | Reserved  |               Not Specified
>    1111b  |           |
>
-------+-----------+------------------------------------------------
> -
RDMA Read with Local Invalidate does not affect the wire. The 'must
invalidate' state is kept in the RNIC that issues the RDMA Read
Request...
> 
> 
> 
> I want to take this opportunity to also mention that the RPC/RDMA
> client-server
> exchange does not support remote-invalidate currently. Because of the
> multiple
> stags supported by the rpcrdma chunking header, and because the client
> needs
> to verify that the stags were in fact invalidated, there is
significant
> overhead,
> and the jury is out on that benefit. In fact, I suspect it's a lose at
> the client.
> 
> Tom (Talpey).
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit
http://openib.org/mailman/listinfo/openib-
> general