[ofa-general] [PATCH v3] iw_cxgb3: Support "iwarp-only" interfacesto avoid 4-tuple conflicts.

[Tom Tucker]

On Mon, 2007-09-24 at 16:30 -0500, Glenn Grundstrom wrote:
>  
> > -----Original Message-----
> > From: Roland Dreier [mailto:rdreier at cisco.com] 
> > Sent: Monday, September 24, 2007 2:33 PM
> > To: Glenn Grundstrom
> > Cc: Steve Wise; sean.hefty at intel.com; general at lists.openfabrics.org
> > Subject: Re: [ofa-general] [PATCH v3] iw_cxgb3: Support 
> > "iwarp-only" interfacesto avoid 4-tuple conflicts.
> > 
> >  > I'm sure I had seen a previous email in this thread that 
> > suggested using
> >  > a userspace library to open a socket
> >  > in the shared port space.  It seems that suggestion was 
> > dropped without
> >  > reason.  Does anyone know why?
> > 
> > Yes, because it doesn't handle in-kernel uses (eg NFS/RDMA, 
> > iSER, etc).
> 
> The kernel apps could open a Linux tcp socket and create an RDMA
> socket connection.  Both calls are standard Linux kernel architected
> routines. 

This approach was NAK'd by David Miller and others...

>  Doesn't NFSoRDMA already open a TCP socket and another for 
> RDMA traffic (ports 2049 & 2050 if I remember correctly)?  

The NFS RDMA transport driver does not open a socket for the RDMA
connection. It uses a different port in order to allow both TCP and RDMA
mounts to the same filer.

> I currently
> don't know if iSER, RDS, etc. already do the same thing, but if they
> don't, they probably could very easily.
> 

Woe be to those who do so...

> > 
> > Does the neteffect NIC have the same issue as cxgb3 here?  What are
> > your thoughts on how to handle this?
> 
> Yes, the NetEffect RNIC will have the same issue as Chelsio.  And all
> Future RNIC's which support a unified tcp address with Linux will as
> well.
> 
> Steve has put a lot of thought and energy into the problem, but
> I don't think users & admins will be very happy with us in the long run.
> 

Agreed.

> In summary, short of having the rdma_cm share kernel port space, I'd
> like to see the equivalent in userspace and have the kernel apps handle
> the issue in a similar way as described above.  There are a few
> technical
> issues to work through (like passing the userspace IP address to the
> kernel),

This just moves the socket creation to code that is outside the purview
the kernel maintainers. The exchanging of the 4-tuple created with the
kernel module, however, is back in the kernel and in the maintainer's
control and responsibility. In my view anything like this will be viewed
as an attempt to sneak code into the kernel that the maintainer has
already vehemently rejected. This will make people angry and damage the
cooperative working relationship that we are trying to build.

>  but I think we can solve that just like other information that
> gets passed from user into the IB/RDMA kernel modules.
> 


Sharing the IP 4-tuple space cooperatively with the core in any fashion
has been nak'd. Without this cooperation, the options we've been able to
come up with are administrative/policy based approaches. 

Any ideas you have along these lines are welcome.

Tom

> Glenn.
> 
> > 
> >  - R.
> > 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general