[ofa-general] Bugs in opensm/libvendor

[Todd Rimmer]

> From: general-bounces at lists.openfabrics.org [mailto:general-
> bounces at lists.openfabrics.org] On Behalf Of Sasha Khapyorsky
> Sent: Monday, December 15, 2008 10:39 AM
> >
> > That's a good question - and I'm going to ask around and double check.
> > My first reaction was that you have to specify how many paths you want
> > from the query - but you're right, the spec doesn't say that.
> 
> Yes, it looks like this (but I cannot understand "why" :( ). But even more
> strange (IMHO) limitation is mandatory SGID - actually it should make
> illegal such GetTable queries as all-to-all, SLID-to-all, etc.. I
> thought that it is permitted.
[Todd Rimmer] It's about scalability.  An all to all query in a fabric would return at least N^2 path records, add in an LMC, some varied SLs and PKeys and quickly it becomes ludicrously large.  For example at only 25 nodes, with 8 SLs and 4 PKeys you could have 20,000 path records returned.  With 1000 nodes and 1 SL and 1 PKey you have 1,000,000 path records.

The practical use of GetTable(PathRecord) is for a node to find a path to another node, hence SGID limitation makes sense.

> 
> > I'm going to do some research on my end. Are you saying that
> > IB_MAD_ATTR_PATH_RECORD should only ever return a single path?
> 
> With GetTable? I think it shouldn't (for some queries it will - such as
> SLID + DLID).
[Todd Rimmer] With GetTable the NumbPath parameter is required and sets a cap on the number of returned paths per SGID-DGID combination.  This makes sense since most applications are only prepared to use a handful of paths to each destination.  Many would only use 2 paths for a failover scheme.  Those which only need 1 path can use Get(PathRecord) in which case NumbPath is implied to be 1.

For scalability, it's best to avoid queries which don't specify a destination (eg. always provide DGID, DLID, etc).  Queries without a destination can still be huge and consume a lot of SA and memory resources to process (eg. at 1000 nodes LMC=2, 8 SLs you get 32000 path records back).  Besides its very rare than an application really wants to know the path to EVERY other node.

Todd Rimmer