[openib-general] [PATCH 6/6] [RFC] iser socket

[Caitlin Bestler]

Alexander Nezhinsky wrote:
> Caitlin Bestler wrote:
> 
>>> So what do the iser sockets do?  They look like noop stubs to me.
>>> 
>> 
>> Good question.
>> 
>> I am guessing that they are exactly noop stubs, and the real point is
>> to have a socket associated with an iSER RDMA connection.
>> 
>> My question is why? Unless the attempt is to allow upgrading an iSCSI
>> stream connection to an iSER connection I don't see why an iSER RDMA
>> connection is in any more of a need for having a proxy socket than
>> any other RDMA connection. 
>> 
> We don't upgrade iSCSI stream connection but start with an
> RDMA connection right away.
> The iSER code is going to be one of open-iscsi transports,
> and open-iscsi opens connections using sockets from user
> space, which is only natural with tcp.
> The iSER RC connection should be open from kernel, so this
> special socket gives us an opportunity to do so, while
> leaving intact the entire mechanism of connection
> establishment and user-kernel handover.
> We don't really need to implement read/write primitives
> because they are initiated either from within kernel
> transport module itself or through a special user-kernel
> interface bypassing the socket.
> 
>> I really don't see the benefit of having a "socket" that is not truly
>> integrated with the host stack. What socket attributes are being
>> sought? And how is it unique to iSER as opposed to RDMA in general?
>> 
> Perhaps it is plausible to implement a general-purpose
> stack-integrated socket giving access to IB RC connections,
> if this is what you mean.
> But this was clearly out of scope for the iSER initiator.
> We sought a solution to the immediate problem in open-iscsi.
> So the main socket feature used here was a neat way to
> delegate the IB connection establishment from user space to kernel.

I'm more questioning what is the purpose of opening a "socket"
from user-space when the only thing you can do with it is 
hand it off to a specialized kernel component.

There really isn't a lot of user-space logic being preserved
here unless you keep the negotiations in user-space as well.
A socket handle that can only be used to hand off a connection
doesn't seem to accomplish that. It also raises a major problem
for iSER/IP, in that there would be a reasonable expectation
that the actual TCP state was being transferred with the socket.
The consensus within netdev is clearly against allowing dependencies
on TCP state internals. I happen to disagree with that consensus,
but I still think we should make a good faith effort to comply
with it.

I think a CMA style interface is more what is called for here.
CMA hides the pre-RDMA communications, and provides a one-shot
message exchange service to enable configuring RDMA.

An equivalent service for iSCSI/iSER would establish an
"iSCSI/iSER" connection to the target, and then enable
exchange of "startup phase" messages before transitioning
to either iSCSI full featured phase, iSER full featured
phase, or terminating the connection.

Using iscsi_tcp the "startup phase" messages would simply
be translated into writes/reads over a TCP socket. For
iSER/IB they would be translated to Send Messages. For
iSER/IP they could be translated into TCP segments.

Exposing a "socket" that has nothing resembling the functionality
of a socket doesn't strike me as a good solution. An approach
that is more consistent with iWARP connection setup, as I
outlined above, would seem to make more sense.




Why isn't a CMA style interface more applicable here?