<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=windows-1251"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
terry watson a écrit :
<blockquote cite="mid:BAY117-W4901338B668F9C424E63ECDBE50@phx.gbl"
type="cite">
<pre wrap="">Hi all,
I will be performing some testing of partitioning used as a security control. Am I right in believing that IBIS will be able to set partition table values of the local compute node I am logged on to, even though they are not using OpenSM, but rather a SM on a switch? Could I then attempt to access a partition that I was originally excluded from accessing?
I am new to Infiniband technology and would also appreciate a response from an expert who has views on the strength of the security that partitioning provides in separating two clusters that should have no interaction whatsoever.
Thanks,
Dave
_________________________________________________________________
Discover the new Windows Vista
<a class="moz-txt-link-freetext" href="http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE_______________________________________________">http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE_______________________________________________</a>
general mailing list
<a class="moz-txt-link-abbreviated" href="mailto:general@lists.openfabrics.org">general@lists.openfabrics.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general">http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general</a>
To unsubscribe, please visit <a class="moz-txt-link-freetext" href="http://openib.org/mailman/listinfo/openib-general">http://openib.org/mailman/listinfo/openib-general</a>
</pre>
</blockquote>
The partitions are only managed by the subnet manager - either opensm
running on a node into the fabric or an embedded subnet manager on a
switch.<br>
For opensm , partitions are defined into a configuration file
/etc/opensm/partitions.conf, for a embedded subnet manager, you have to
configure the partitions using the CLI or GUI provided by the switch.<br>
Defining a partition is mainly choosing a pkey and ports nodes with
their membership (limited or not).<br>
<br>
The subnet manager assigned the pkeys to the ports of the node when ib
kernel modules are loaded. You can see the partitions the IB port
belong to by ( I mean those defined by the subnet manager) :
<br>
# grep -v 0x0000 <i class="moz-txt-slash"><span class="moz-txt-tag">/</span>sys/class/infiniband/mthca0/ports/1/pkeys<span
class="moz-txt-tag">/</span></i>*
<br>
/sys/class/infiniband/mthca0/ports/1/pkeys/0:0xffff
<br>
/sys/class/infiniband/mthca0/ports/1/pkeys/1:0x8001
<br>
/sys/class/infiniband/mthca0/ports/1/pkeys/2:0x8002
<br>
/sys/class/infiniband/mthca0/ports/1/pkeys/3:0x8003
<br>
/sys/class/infiniband/mthca0/ports/1/pkeys/4:0x8010
<br>
<br>
A port may belong to many partitions. Nodes (ports) may have different
partitions configurations. Partitions order for a port is not always
the same ( it may depend on the chronology of partition declarations in
the subnet manager)<br>
<br>
Over these partitions, you can define new IP (IP over IB) interfaces by
creating files like /etc/sysconfig/network-scripts/ifcfg-ib0.8002 :<br>
<tt># cat /etc/sysconfig/network-scripts/ifcfg-ib0.8002
<br>
DEVICE=ib0.8002
<br>
BOOTPROTO=static
<br>
IPADDR=XXX.YYY.ZZZ.TTT
<br>
NETMASK=255.255.255.0
<br>
NETWORK=255.255.255.0
<br>
ONBOOT=yes
</tt><br>
<br>
The openibd script create the child interface and configure it at
system startup using some special devices to do that :<br>
echo $pkey > /sys/class/net/ib0/create_child<br>
<br>
But this command creates only a child interface on the node, but
communications on this interface will not work until you add the port
node to the corresponding partition into the subnet manager
configuration. Then you will see the pkey appearing automatically into
files <i class="moz-txt-slash"><span class="moz-txt-tag">/</span>sys/class/infiniband/mthca0/ports/1/pkeys<span
class="moz-txt-tag">/</span></i>*
on the node.<br>
<br>
<tt>[root@cors118 ~]# echo 0x8009 > /sys/class/net/ib0/create_child
<br>
[root@cors118 ~]# dmesg | grep 8009
<br>
divert: not allocating divert_blk for non-ethernet device ib0.8009
<br>
[root@cors118 ~]# grep -v 0x0000 <i class="moz-txt-slash"><span
class="moz-txt-tag">/</span>sys/class/infiniband/mthca0/ports/1/pkeys<span
class="moz-txt-tag">/</span></i>*
<br>
/sys/class/infiniband/mthca0/ports/1/pkeys/0:0xffff
<br>
/sys/class/infiniband/mthca0/ports/1/pkeys/1:0x8001
<br>
/sys/class/infiniband/mthca0/ports/1/pkeys/2:0x8002
<br>
/sys/class/infiniband/mthca0/ports/1/pkeys/3:0x8003
<br>
/sys/class/infiniband/mthca0/ports/1/pkeys/4:0x8010
<br>
[root@cors118 ~]# ifconfig -a | grep 8009
<br>
ib0.8009 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
<br>
[root@cors118 ~]# echo 0x8009 > /sys/class/net/ib0/delete_child
<br>
[root@cors118 ~]# dmesg | grep 8009
<br>
divert: not allocating divert_blk for non-ethernet device ib0.8009
<br>
divert: no divert_blk to free, ib0.8009 not ethernet
</tt><br>
<br>
To use MPI with partitions, you have also to configure it (in the
configuration file) . For MVAPICH you must use VIADEV_DEFAULT_PKEY_IX
or VIADEV_DEFAULT_PKEY in the config file :
/usr/mpi/gcc/mvapich-1.0.0/etc/mvapich.conf . AT CEA, I'm using
VIADEV_DEFAULT_PKEY (pkey value)<br>
as we have nodes with different partitions configurations.<br>
<br>
<br>
Hoping this will help you.<br>
Regards<br>
Philippe Gregoire CEA/DAM<br>
<br>
<br>
</body>
</html>