[ofa-general] Re: IPoIB path caching

[Jason Gunthorpe]

On Thu, Aug 02, 2007 at 11:37:27AM -0700, Sean Hefty wrote:
> >Well, I still think the simplest thing is to make a new netlink
> >protocol to maintain a cache table in the kernel and then a simple
> >user space program that uses the user space RMPP interface (and trap
> >subscription..) to do GetTable queries and uses netlink to groom the
> >kernel cache. Basically exactly what you have now but seperated into
> >two halfs.
> 
> This makes more sense to me.  I thought you were suggesting that the 
> entire implementation be in user space.

Well, both really..

The kernel side table could either include all possible PRs, or some
subset. Your implementation uses all possible PRs today and that is
fine, but to support QOS or routers you might want to only have a
subset resident in the kernel at any time. The only practical impact
this has on the kernel support is that cache lookup misses should have
a way to be passed to userspace for additional processing (if
userspace requests that).
 
> >The main thing is to pick the kernel's table structure so that it can
> >be extended to support future things like multipath, QOS and routers..
> 
> Multipath is already supported, but I'm storing path records, not 
> creating them from tables currently.  QoS support is a more immediate 
> concern, but I believe we can have a fairly simple set of tables to 
> support that.  Routers will present more of a challenge.

I am imagining the kernel being very simple, take the standard IB PR
fields and match them against a caching table. If the matching misses
pass it to userspace, or do a PR. If the QOS tables cannot be
expressed as simple matching then the userspace part would have to do
the table based calculations to match it and populate the kernel cache
table.

Jason