[openib-general] IB Address Translation service

[Yaron Haviv]

> -----Original Message-----
> From: openib-general-bounces at openib.org [mailto:openib-general-
> bounces at openib.org] On Behalf Of Roland Dreier
> Sent: Monday, February 28, 2005 7:13 PM
> To: shaharf
> Cc: openib-general at openib.org
> Subject: Re: [openib-general] IB Address Translation service
> 
> This API seems overly complex and at the same time too inflexible to
> me.  However, rather than getting bogged down nitpicking about APIs, I
> think we have to take a few steps back.

I believe the API is very flexible, but we are pretty open to here what
you think is needed in addition 

> First, let's understand the problem we're trying to solve.  Who are
> the consumers of this address translation service?

The first problem is that most ULPs use valid IP addresses for
simplicity (DAPL, iSER, NFS/RDMA, SDP, MPI, etc') and someone needs to
resolve it to an IB address and device to use IB. This should take into
account cases where there are more than one HCAs in the system.
Preferable/optionally the ULP would like to know which partition to use
if there is more than one, and leverage on the IP subnetting done by
IPoIB.

It is possible to replicate the same code you have in SDP (which is also
not complete) across all ULP's, I assume a better way is to provide it
in one central place.
There are also two proposed address resolution mechanisms, one is ARP
used by SDP, and one is ATS used by some DAPL consumers, and we believe
it is better to combine them under the same API.

The second problem relates to mapping of IB GID to one or more Path
records
This is also something needed for ALL ULP's. today each ULP provides the
minimal subset of path resolution functionality without taking into
account topics such as partitioning, QoS, source routing and
multi-pathing.
Some of these require using special SA queries (such as SA Multipath
Record query and QoSPath Query).
I don't think it make sense to put all this functionality into each ULP
as well.

Than we can also discuss, does it make sense to have each path
resolution call lead us to the sa, or does it make more sense to cache
those paths.
And if we cache, doesn't it make more sense to cache/invalidate the
routes to all ULP's rather implementing/having it in each ULP.
Also not sure how a 1000 node cluster functions without the caching.
 
And the last problem is related to reverse resolution from IB to IP
addresses that is needed for DAPL, as well as for different management
and diagnostic tools that want to know what is really that node/port
behind that GID addresses.

So how would you suggest to go about it ?
Duplicate all of that in each ULP ?
Refrain from implementing advanced routing, partitioning, QoS (we cant
really maintain all that advanced code for each ULP) ? 

Our idea is to provide those few helper functions that enable people to
make full use of IB and its features without reading all the IB spec,
and a Phd.
If you clear all the remarks from the library, you will see it is very
slim, and for my understanding includes all the relevant input and
output parameters for each of the 3 functions I mentioned.

As shahar mentioned, this is just a proposal, and if you see any thing
missing in the API, or a better way to address the requirements I just
listed, I'm happy to here.

The API doesn't define the implementation, which we can discuss once we
agree on the functionality and interfaces, and you have some valid
questions there needs to be addressed.

Yaron
 
> Second, let's come up with the right architecture to solve the
> problem.  Are we implementing a library in userspace or a kernel
> module?  Do we have a single cache or do we need multiple caching
> policies?  And so on...
> 
> Finally, we can design the API.
> 
> Thanks,
>   Roland
> 
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit
http://openib.org/mailman/listinfo/openib-
> general