[ofa-general] Re: tcp/rdma port unification - iscsi/ddp/toe vs iwarp/ddp/toe

[Steve Wise]

Or Gerlitz wrote:
> Steve Wise wrote:
>   
>> The iscsi offload drivers all maintain their own private ip addresses
>> hidden from the native stack.  That's how they keep the 4-tuple unique.
>>     
>
> OK, so if the ip address wasn't private then the iscsi offload arp flow
> would have been the same as the iwarp one? and also the other way around,
>   

Yes.

> an iwarp stack can achieve the same impact (unique 4-tuple) either with shared
> port space or with private ip address - and actually, this ip address need not
> be private - it can be just --different-- from the one used for OS TCP.
>   

Yes.  If you don't hide it from the stack, then there is no way (without 
changing the kernel stack code) to keep the stack from using it.  If you 
look back in the netdev/openib mailing lists, you'll see a very painful 
patch set I posted early on to do this sort of scheme with cxgb3.  I 
created alias interfaces with ip addresses that were on their own subnet 
and the iwarp driver would only use those addresses.   The downside of 
this patch was that its up to the sysadmin to configure everything 
correctly.  And there was nothing to stop a TCP app from using those 
addresses (other than admin policy).  Also it duplicates all the 
management of interfaces.  Its a bad solution IMO.  Roland also resisted 
admitting that solution because of these downsides. 

But it did resolve the issue by forcing iwarp connections on their own 
ip addresses.  :)

Here was version 3 of the patch from 2007: 

http://thread.gmane.org/gmane.linux.drivers.openib/44463/focus=73155


For iSCSI, its perhaps a little easier because you have only one 
application: iSCSI.  But the original iSCSI offload submission from 
chelsio was to use whatever ip addresses were bound to the NIC interface 
and use the native neighbor discovery logic like rdmacm does.  That was 
rejected by the netdev maintainers.


> Anyway, bottom line, I see that you and Karen have chosen different approaches
> for the solution of the same problem (over the same HW...) and special reason
> for that - or it was a matter of how things were evolving? 
>
>   

More a matter of what could advance upstream.


Steve.