[openib-general] iser/uverbs integration

[Gleb Natapov]

Hello,

We've encountered a problem with iSCSI/iser/uverbs integration so I decided to 
move this discussion here since the problem is more IB oriented.

The way TCP iSCSCI works is this: connection entirely happens in userspace and
then connected fd is passed to kernel via netlink and from that point on the 
socket is used from the kernel part of iSCSI for actual data transfer.

Obviously it is desirable to keep the same semantic for IB too and it is not 
possible to use userspace QP from the kernel in the current implementation.
Our proposition was to create new socket type. This socket will use new 
connection API from inside the kernel to connect to the target. When fd will be
transfered from userspace to kernel iser will use already connected QP for data
transfer. If Yaron's proposition (WarpoverIB.txt) to use CM private data for 
transferring IP info on connect will be accepted the socket will event provide 
getpeername() functionality that is needed by NFS to properly support /etc/exports.

The response for this proposition you can see in the forwarded mail at the end.

I looked into openIB code to see what it'll take to use userspace QP from the
kernel and it doesn't looks good. All resources needed to use QP belong to 
userspace (uar, qp buffer, cq buffer). In order to use QP from the kernel we
will need to write data directly into the user pages and this is ugly and may 
be slow (what if page is in HIGHMEM?).

The question is what is the best way to proceed? Will the changes needed to
use userspace QP from kernel will be accepted? How NFS/RDMA works now?


-----Original Message-----
From: open-iscsi at googlegroups.com [mailto:open-iscsi at googlegroups.com] On Behalf Of Christoph Hellwig
Sent: Tuesday, August 23, 2005 1:02 PM
To: open-iscsi at googlegroups.com
Subject: Re: Connect/Disconnect for iscsi_iser transport


On Tue, Aug 23, 2005 at 08:50:12AM +0300, Erez Zilber wrote:
>
> Hi,
>
> As I understand it, one of the ideas in open-iscsi is handling connect/disconnect 
> issues in user space. This may be problematic for iscsi_iser transport since 
> the ib-verbs implementation doesn't allow the transfer of user space qp to 
> kernel space (which is required if we want to connect from user space and then 
> use the same connection from kernel space). Therefore, I think that one of the 
> following soultions may be suitbale for this problem(these solutions are ok for
> other transports as well):

Please implement that support in the uverbs then.  It'll probably make
sense for things like nfs over ib aswell where we can move the
connection establishment into the mount helper.

There's no way the iscsi code is going to grow ioctls, and your socket
approach is ugly as hell aswell.

--
			Gleb.