[ofa-general] Re: [openSM] Pkey index and Pkey value

[Hal Rosenstock]

On Fri, 2007-11-16 at 17:37 +0100, Philippe Gregoire wrote:
> Keshetti Mahesh a écrit :
> >> No; there is no requirement for a pkey value to be in the same index
> >> in different ports.
> >>     
> >
> > But if you want to run an MPI application in all the ports belonging to
> > one partition (e.g: partition2 in my previous exmple) a conflict will arise
> > as most of the MPI implementations have provision of allowing only one
> > Pkey index (e.g: ib_pkey_ix in openMPI). This problem can be easily solved
> > by storing pkey value at the same pkey index in all ports belonging to same
> > partition.
> >
> > -Mahesh
> > _______________________________________________
> > general mailing list
> > general at lists.openfabrics.org
> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >
> > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
> >
> >   
> No, some nodes in the cluster may have a specific partitions configuration
> so for the same partition  (ex 0x8088) pkey index may change on nodes.
> 
> For more informations,
> See https://bugs.openfabrics.org/show_bug.cgi?id=519
> 
> That's why a new configuration parameter has been created for mvapich :
> # grep -C2 VIADEV_DEFAULT_PKEY /usr/mpi/gcc/mvapich-0.9.9/etc/mvapich.conf
> # VIADEV_DEFAULT_MAX_RDMA_DST_OPS
> #-----------------------------------------------------------------------------------------
> # VIADEV_DEFAULT_PKEY_IX
> #
> # Default value:none
> # Index of predefined partition
> #-----------------------------------------------------------------------------------------
> # VIADEV_DEFAULT_PKEY
> #
> # Default value:none
> # Name (key) of predefined partition. One can pass it both in decimal 
> end hexadecimal
> # format. This parameter prevail over VIADEV_DEFAULT_PKEY_IX, when defined.
> #-----------------------------------------------------------------------------------------
> 
> I'm wondering if the ib_pkey_val parameter has not the same goal for 
> OPENMPI :
> 
> see https://svn.open-mpi.org/trac/ompi/changeset/14463

I recently discussed this with Jeff Squyres and he showed me a recent
checkin (not sure for which OpenMPI version) which supports either pkey
value or index.

So it looks like both OpenMPI and MVAPICH support this. Not sure about
any commercial MPIs.

Bottom line, pkey value will always work; pkey index is SM dependent.

-- Hal

> Philippe
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general