[ofa-general] Bugs in opensm/libvendor

[Sasha Khapyorsky]

Hi Todd,

On 06:39 Tue 16 Dec     , Todd Rimmer wrote:
> > From: general-bounces at lists.openfabrics.org [mailto:general-
> > bounces at lists.openfabrics.org] On Behalf Of Sasha Khapyorsky
> > Sent: Monday, December 15, 2008 10:39 AM
> > >
> > > That's a good question - and I'm going to ask around and double check.
> > > My first reaction was that you have to specify how many paths you want
> > > from the query - but you're right, the spec doesn't say that.
> > 
> > Yes, it looks like this (but I cannot understand "why" :( ). But even more
> > strange (IMHO) limitation is mandatory SGID - actually it should make
> > illegal such GetTable queries as all-to-all, SLID-to-all, etc.. I
> > thought that it is permitted.
> [Todd Rimmer] It's about scalability.  An all to all query in a fabric would return at least N^2 path records, add in an LMC, some varied SLs and PKeys and quickly it becomes ludicrously large.  For example at only 25 nodes, with 8 SLs and 4 PKeys you could have 20,000 path records returned.  With 1000 nodes and 1 SL and 1 PKey you have 1,000,000 path records.
> 
> The practical use of GetTable(PathRecord) is for a node to find a path to another node, hence SGID limitation makes sense.

Of course scalability is important, but I'm not sure that discussed
limitations are real solution - if some port will want to get all-to-all
paths information anyway it just will send much more queries and
potentially will hurt SA scalability and fabric traffic even more.

> > > I'm going to do some research on my end. Are you saying that
> > > IB_MAD_ATTR_PATH_RECORD should only ever return a single path?
> > 
> > With GetTable? I think it shouldn't (for some queries it will - such as
> > SLID + DLID).
> [Todd Rimmer] With GetTable the NumbPath parameter is required and sets a cap on the number of returned paths per SGID-DGID combination.  This makes sense since most applications are only prepared to use a handful of paths to each destination.  Many would only use 2 paths for a failover scheme.  Those which only need 1 path can use Get(PathRecord) in which case NumbPath is implied to be 1.

Sure. But whole discussion is about GetTable, not just Get (this case is
clear for me).

> For scalability, it's best to avoid queries which don't specify a destination (eg. always provide DGID, DLID, etc).  Queries without a destination can still be huge and consume a lot of SA and memory resources to process (eg. at 1000 nodes LMC=2, 8 SLs you get 32000 path records back).  Besides its very rare than an application really wants to know the path to EVERY other node.

As I described in another email example, let look at practical SGID/DGID
case when LMC=15 - this will require at least 256 records, and there is
no (known for me) easy way to get this in IBA complaint way when
NumbPath is limited by 7 bits.

Sasha