[Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMA APIs and ULPs for Linux

[Bernard Metzler]

Sukanta,

without touching any TOE issues (this question is about RDMA, right?), 
after
transforming a TCP connection into RDMA mode and using an RDMA API,
the socket file descriptor is not longer to be used for communication.
In fact, on some implementations the stream socket resources will
even get released. That is, if the socket was the direct consumer
of the TCP stream, then now it is RDMAP/DDP/MPA.
RDMA APIs such as IT-API defining a specific call to convert a socket 
based
connection into RDMA mode (e.g., it_socket_convert()).
Other consumers may directly start via an RDMA API, never
opening a consumer controlled socket.

So, in RDMA mode, communication will happen via the RDMA API. At this
stage, the kernel still have to keep completely in its hands the
synchronisation of state information related to that offloaded
connection(s) with the host stack (it would have to protect the
local port used by the offloaded connection for example, others
are routing, ARP, SNMP...), but it is not involved at the data path.

With respect to the kernel based TCP stack, what is not needed is
a hack into the stack and scatter/gather state information of the live
TCP connection between kernel and RNIC, but to find one clean interface
to transfer state information out of that stack and to the RNIC.


With limited benefit, one could of course also implement native 
sockets over RDMA, where an in-kernel midlayer on top of kernel
RDMA Verbs is doing the translation between send(), receive() to 
post_send, post_receive. But usage of 'true RDMA' operations
like RDMA READ or WRITE might be limited, and I don't see much value 
for the user here. One variety of this approach with less limited
access to RMDA benefits might be sockets with extended RDMA 
semantics.

Bernard.

rdma-developers-admin at lists.sourceforge.net wrote on 27.05.2005 15:40:43:

> Venkata,
>    How will that work? If the RNIC offloads RDMA and
> TCP completely from the Operating System and does not
> share any state information then the application
> running on the host will never be in the position to
> utilize the socket interface to use the communication
> logic to send and receive data between the remote node
> and itself. Some information needs to be shared. How
> much of it and what exactly needs to be shared is the
> question.
> 
> Thanks
> SG
> 
> --- Venkata Jagana <jagana at us.ibm.com> wrote:
> > 
> > 
> > 
> > 
> > 
> > 
> > rdma-developers-admin at lists.sourceforge.net wrote on
> > 05/25/2005 09:47:00
> > PM:
> > 
> > > Venkata,
> > > Interesting coincidence: I was talking with
> > someone (at HP) today
> > > who knows substantially more than I do about
> > RNICs.
> > > They indicated RNICs need to manage TCP state on
> > the card from userspace.
> > > I suspect that's only possible through a private
> > interface
> > > (e.g. ioctl() or /proc) or the non-existant (in
> > kernel.org)
> > > TOE implementation. Is this correct?
> > >
> > 
> > Not correct.
> > 
> > Since RNICs are offloaded adapters with RDMA
> > protocols layered on
> > top of TCP stack, they do maintain the TCP state
> > internally but
> > it does not expose to the host. RNIC expose only
> > RNIC Verbs interface
> > to the host bot not TOE interface.
> > 
> > Thanks
> > Venkat
> > 
> > >
> > > hth,
> > > grant
> > >
> > >
> > >
> >
> -------------------------------------------------------
> > > SF.Net email is sponsored by: GoToMeeting - the
> > easiest way to
> > collaborate
> > > online with coworkers and clients while avoiding
> > the high cost of travel
> > and
> > > communications. There is no equipment to buy and
> > you can meet as often as
> > > you want. Try it
> >
> free.http://ads.osdn.com/?ad_id=7402&alloc_id=16135&op=click
> > > _______________________________________________
> > > Rdma-developers mailing list
> > > Rdma-developers at lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/rdma-developers
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around 
> http://mail.yahoo.com 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by Yahoo.
> Introducing Yahoo! Search Developer Network - Create apps using Yahoo!
> Search APIs Find out how you can build Yahoo! directly into your own
> Applications - visit 
http://developer.yahoo.net/?fr=offad-ysdn-ostg-q22005
> _______________________________________________
> Rdma-developers mailing list
> Rdma-developers at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/rdma-developers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050527/278eacf8/attachment.html>