[ofa-general] why netwoked file system(e.g. nfs, pvfs, etc.) supported IB by using access layer (linux kernel ib ops)

[Talpey, Thomas]

At 03:14 AM 6/20/2007, ncic wrote:
>why didn't they support ib with sdp?

There are two main answers.

The first is licensing. SDP licensing is wrapped up in a Microsoft
intellectual property issue, this has prevented its inclusion in some
kernels, including Linux. So, upper layers cannot depend in its
presence.

The second, speaking for NFS at least, is performance. SDP relies
heavily on additional setup exchanges and RDMA Read for transparency,
these negatively impact performance. With minimal additional work,
the same unmodified upper layer NFS filesystem code can use native
RDMA exchanges via the RPC layer and achieve truly excellent
performance. Check out Helen Chen's presentation from the recent
Sonoma workshop.
<http://www.openfabrics.org/archives/spring2007sonoma/Tuesday%20May%201/Helen%20Chen%20NFS%20over%20RDMA%20-%20IB%20and%20iWARP-5.pdf>

In the NFS case, the protocol is on a standards track and published in
the IETF (I'm the primary author), I'm hopeful that the edits I'm currently
preparing for publication will be finalized around the July meeting. And, we
have complete implementations of both client and server in both Linux and
OpenSolaris.

For transparent mode, don't discount ordinary sockets over a connected
mode IPoIB approach. The performance is very good, and provides a fully
transparent solution to all upper layers. RDMA is better though, by (greatly)
reducing overhead.

Tom.