[ofw] RE: ofw Digest, Vol 15, Issue 12
Slava Strebkov
slavas at voltaire.com
Thu Apr 3 02:59:19 PDT 2008
This is the first version of partition. Specifying pkey value in
registry helps to use only desired partition from the pkey table
obtained from switch.
-----Original Message-----
From: ofw-bounces at lists.openfabrics.org
[mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of
ofw-request at lists.openfabrics.org
Sent: Thursday, April 03, 2008 7:16 AM
To: ofw at lists.openfabrics.org
Subject: ofw Digest, Vol 15, Issue 12
Send ofw mailing list submissions to
ofw at lists.openfabrics.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
or, via email, send a message with subject or body 'help' to
ofw-request at lists.openfabrics.org
You can reach the person managing the list at
ofw-owner at lists.openfabrics.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of ofw digest..."
Today's Topics:
1. RE: [PATCH] partition (Fab Tillier)
----------------------------------------------------------------------
Message: 1
Date: Wed, 2 Apr 2008 22:14:18 -0700
From: Fab Tillier <ftillier at windows.microsoft.com>
Subject: RE: [ofw] [PATCH] partition
To: Yiftah Shahar <yiftahs at voltaire.com>, Slava Strebkov
<slavas at voltaire.com>, "ofw at lists.openfabrics.org"
<ofw at lists.openfabrics.org>
Message-ID:
<52F2F0099A281049A606C312906481CE02C7CD3DF4 at NA-EXMSG-W601.wingroup.winde
ploy.ntdev.microsoft.com>
Content-Type: text/plain; charset="iso-8859-1"
If you don't trust whoever controls the SM you have a serious problem on
your hands. A single point of control is much preferable to thousands
of them. A thousand duplicated points of control is error prone.
-Fab
-----Original Message-----
From: Yiftah Shahar [mailto:yiftahs at voltaire.com]
Sent: Wednesday, April 02, 2008 10:06 PM
To: Fab Tillier; Slava Strebkov; ofw at lists.openfabrics.org
Subject: RE: [ofw] [PATCH] partition
Hi Fab,
> This way all configuration is automatic, rather than requiring
registry
> modification. A single point of configuration (the SM) takes care of
> it all.
>From our experience with many different types of customers the
requirement is clear: whenever someone does something on the switch,
NOTHING should automatically happen on the host.
This is a HUGE security hole and sys admins will complain that the
networking group who controls the switches is now able to screw with
host configuration from the switch without explicitly having been
permitted to do so by someone on the windows sysadmin security team.
Maybe we need to have other control mechanisms but for first version
registry key is reasonable way to go.
Do remember that we still have this completion with error bug that with
out it this feature is not relevant ( ;-( ) so if anyone can help us fix
it this will be great.
Yiftah
________________________________________
From: ofw-bounces at lists.openfabrics.org
[mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Fab Tillier
Sent: Wednesday, April 02, 2008 17:30
To: Slava Strebkov; ofw at lists.openfabrics.org
Subject: RE: [ofw] [PATCH] partition
Why not have ibbus create a PDO for each valid pkey in the pkey table
automatically, and then report the pkey as part of the ipoib_ifc_data_t
structure returned when IPoIB queries for the IBAL_INTERFACE_STANDARD?
Note that a similar change could then be done to the other interfaces
reported to allow partition management for all IB attached devices.
This way all configuration is automatic, rather than requiring registry
modification. A single point of configuration (the SM) takes care of it
all.
Also, it might be clearer if you renamed the port_guid structure to
port_info or similar, so you don't have port_guid.port_guid and
port_guid.pkey.
-Fab
From: ofw-bounces at lists.openfabrics.org
[mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Slava Strebkov
Sent: Wednesday, April 02, 2008 5:39 AM
To: ofw at lists.openfabrics.org
Subject: [ofw] [PATCH] partition
Hi,
The following patch implements partition feature. It's still not
working; CQ completes WR's with IB_WCS_WR_FLUSHED_ERR.
Pkey value is written into registry under
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ibbus\Parameters.
To use partition, specify valid pkey value in IbPkey.
Ibbus driver creates additional PDO for each Pkey value read from
registry.
Index: core/bus/kernel/bus_driver.c
===================================================================
--- core/bus/kernel/bus_driver.c (revision 1026)
+++ core/bus/kernel/bus_driver.c (working copy)
@@ -56,6 +56,8 @@
#else
#define DEFAULT_NODE_DESC "OpenIB Windows(r) Host"
#endif
+/* Pkey read from registry */
+pkey_array g_ib_pkey_array;
char node_desc[IB_NODE_DESCRIPTION_SIZE];
@@ -185,7 +187,100 @@
BUS_ENTER( BUS_DBG_DRV );
}
+static void __prepare_pKey_array(const UNICODE_STRING *str)
+{
+ uint16_t i;
+ NTSTATUS status;
+ ANSI_STRING ansi_str;
+ g_ib_pkey_array.num_pKeys = 0;
+ status = RtlUnicodeStringToAnsiString(&ansi_str,str,TRUE);
+ if(! NT_SUCCESS(status))
+ return;
+
+ for (i = 0; (i < ansi_str.MaximumLength) &&
(g_ib_pkey_array.num_pKeys < MAX_NUM_PKEY) ; i++)
+ {
+ switch(ansi_str.Buffer[i])
+ {
+ case '0':
+ if (((i+1) < ansi_str.Length) && ( (
ansi_str.Buffer[i+1] == 'x') || ( ansi_str.Buffer[i+1] == 'X')))
+ break;
+ else
+ {
+
g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] = \
+
(g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] << 4)| 0;
+ break;
+ }
+
+ case 'x':
+ case 'X':
+ break;
+
+ case ',':
+ g_ib_pkey_array.num_pKeys++;
+ break;
+
+ case 'A':
+ case 'a':
+
g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] = \
+
(g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] << 4)| 0xA;
+ break;
+
+ case 'B':
+ case 'b':
+
g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] = \
+
(g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] << 4)| 0xB;
+ break;
+
+ case 'C':
+ case 'c':
+
g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] = \
+
(g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] << 4)| 0xC;
+ break;
+
+ case 'D':
+ case 'd':
+
g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] = \
+
(g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] << 4)| 0xD;
+ break;
+
+ case 'E':
+ case 'e':
+
g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] = \
+
(g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] << 4)| 0xE;
+ break;
+
+ case 'F':
+ case 'f':
+
g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] = \
+
(g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] << 4)| 0xF;
+ break;
+
+ case '1':
+ case '2':
+ case '3':
+ case '4':
+ case '5':
+ case '6':
+ case '7':
+ case '8':
+ case '9':
+
g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] = \
+
(g_ib_pkey_array.pKey[g_ib_pkey_array.num_pKeys] << 4)|
(ansi_str.Buffer[i] - '0');
+ break;
+
+ case '\0':
+ g_ib_pkey_array.num_pKeys++;
+ break;
+
+ default:
+ break;
+
+ }
+ }
+
+ RtlFreeAnsiString(&ansi_str);
+}
static NTSTATUS
__read_registry(
IN
UNICODE_STRING* const p_registry_path )
@@ -194,6 +289,7 @@
/* Remember the terminating entry in the table below. */
RTL_QUERY_REGISTRY_TABLE table[9];
UNICODE_STRING
param_path;
+ UNICODE_STRING
pkeyString;
BUS_ENTER( BUS_DBG_DRV );
@@ -212,7 +308,18 @@
RtlAppendUnicodeStringToString( ¶m_path, p_registry_path
);
RtlAppendUnicodeToString( ¶m_path, L"\\Parameters" );
+ RtlInitUnicodeString(&pkeyString,NULL);
+ pkeyString.MaximumLength = 1024*sizeof(WCHAR);
+ pkeyString.Buffer = cl_zalloc( pkeyString.MaximumLength );
+ if( !pkeyString.Buffer )
+ {
+ cl_free( param_path.Buffer );
+ BUS_TRACE_EXIT( BUS_DBG_ERROR,
+ ("Failed to allocate parameters path
pkeyString.\n") );
+ return STATUS_INSUFFICIENT_RESOURCES;
+ }
+
/*
* Clear the table. This clears all the query callback
pointers,
* and sets up the terminating table entry.
@@ -277,9 +384,17 @@
table[7].DefaultData = &g_ioc_poll_interval;
table[7].DefaultLength = sizeof(ULONG);
+ table[8].Flags = RTL_QUERY_REGISTRY_DIRECT;
+ table[8].Name = L"IbPkey";
+ table[8].EntryContext = &pkeyString;
+ table[8].DefaultType = REG_SZ;
+ table[8].DefaultData = NULL;
+ table[8].DefaultLength = 1024*sizeof(WCHAR);
/* Have at it! */
status = RtlQueryRegistryValues( RTL_REGISTRY_ABSOLUTE,
param_path.Buffer, table, NULL, NULL );
+ if (NT_SUCCESS(status))
+ __prepare_pKey_array(&pkeyString);
#if DBG
if( g_al_dbg_flags & AL_DBG_ERR )
@@ -291,6 +406,7 @@
g_al_dbg_level,
g_al_dbg_flags));
+ cl_free( pkeyString.Buffer );
cl_free( param_path.Buffer );
BUS_EXIT( BUS_DBG_DRV );
return status;
Index: core/bus/kernel/bus_driver.h
===================================================================
--- core/bus/kernel/bus_driver.h (revision 1026)
+++ core/bus/kernel/bus_driver.h (working copy)
@@ -203,5 +203,11 @@
extern bus_globals_t bus_globals;
+#define MAX_NUM_PKEY 10
+typedef struct
+{
+ uint16_t pKey[MAX_NUM_PKEY];
+ uint8_t num_pKeys;
+}pkey_array;
#endif /* !defined _BUS_DRIVER_H_ */
Index: core/bus/kernel/bus_port_mgr.c
===================================================================
--- core/bus/kernel/bus_port_mgr.c (revision 1026)
+++ core/bus/kernel/bus_port_mgr.c (working copy)
@@ -54,6 +54,8 @@
DEFINE_GUID(GUID_BUS_TYPE_IBA,
0x5a9649f4, 0x101, 0x4a7c, 0x83, 0x37, 0x79, 0x6c, 0x48, 0x8, 0x2d,
0xa2);
+/* IB pKey array from registry */
+extern pkey_array g_ib_pkey_array ;
/*
* Device extension for IPoIB port PDOs.
@@ -62,7 +64,7 @@
{
bus_pdo_ext_t pdo;
- net64_t
port_guid;
+ port_guid_pkey port_guid;
uint32_t n_port;
/* Number of references on the upper interface. */
@@ -512,8 +514,7 @@
p_pdo_ext = PARENT_STRUCT( p_list_item,
bus_pdo_ext_t, list_item );
p_port_ext = (bus_port_ext_t*)p_pdo_ext;
- if( p_pdo_ext->b_present &&
p_pdo_ext->b_hibernating &&
- (p_port_ext->port_guid ==
p_pnp_rec->p_port_attr->port_guid) )
+ if( p_pdo_ext->b_present &&
p_pdo_ext->b_hibernating && (p_port_ext->port_guid.port_guid ==
p_pnp_rec->p_port_attr->port_guid))
{
n_devs++;
break;
@@ -523,7 +524,7 @@
("Skipped PDO for %s: PDO %p, ext
%p, present %d, missing %d, hibernating %d, port_guid %I64x.\n",
p_pdo_ext->cl_ext.vfptr_pnp_po->identity, p_pdo_ext->cl_ext.p_self_do,
p_pdo_ext, p_pdo_ext->b_present,
p_pdo_ext->b_reported_missing,
- p_pdo_ext->b_hibernating,
p_port_ext->port_guid ) );
+ p_pdo_ext->b_hibernating,
p_port_ext->port_guid.port_guid ) );
}
if (n_devs)
@@ -546,7 +547,7 @@
("Found PDO for %s: PDO
%p, ext %p, present %d, missing %d, hibernating %d, port_guid %I64x.\n",
p_pdo_ext->cl_ext.vfptr_pnp_po->identity, p_pdo_ext->cl_ext.p_self_do,
p_pdo_ext,
p_pdo_ext->b_present, p_pdo_ext->b_reported_missing,
-
p_pdo_ext->b_hibernating, p_port_ext->port_guid ) );
+ p_pdo_ext->b_hibernating,
p_port_ext->port_guid.port_guid ) );
}
}
else
@@ -567,9 +568,9 @@
IN
ib_pnp_port_rec_t* p_pnp_rec )
{
NTSTATUS status;
- DEVICE_OBJECT *p_pdo;
bus_port_ext_t *p_port_ext;
-
+ DEVICE_OBJECT *p_pdo[MAX_NUM_PKEY + 1];
+ uint8_t num_pdo;
BUS_ENTER( BUS_DBG_PNP );
if( !bus_globals.b_report_port_nic )
@@ -588,11 +589,15 @@
return status;
}
+ p_port_ext = NULL;
+
+ for (num_pdo = 0; num_pdo < g_ib_pkey_array.num_pKeys + 1;
num_pdo++)
+ {
/* Create the PDO for the new port device. */
status = IoCreateDevice( bus_globals.p_driver_obj,
sizeof(bus_port_ext_t),
NULL, FILE_DEVICE_CONTROLLER,
FILE_DEVICE_SECURE_OPEN |
FILE_AUTOGENERATED_DEVICE_NAME,
- FALSE, &p_pdo );
+ FALSE, &p_pdo[num_pdo] );
if( !NT_SUCCESS( status ) )
{
BUS_TRACE_EXIT( BUS_DBG_ERROR,
@@ -601,13 +606,13 @@
}
/* Initialize the device extension. */
- cl_init_pnp_po_ext( p_pdo, NULL, p_pdo, bus_globals.dbg_lvl,
+ cl_init_pnp_po_ext( p_pdo[num_pdo], NULL, p_pdo[num_pdo],
bus_globals.dbg_lvl,
&vfptr_port_pnp, &vfptr_port_query_txt );
/* Set the DO_BUS_ENUMERATED_DEVICE flag to mark it as a
PDO. */
- p_pdo->Flags |= DO_BUS_ENUMERATED_DEVICE;
+ p_pdo[num_pdo]->Flags |= DO_BUS_ENUMERATED_DEVICE;
- p_port_ext = p_pdo->DeviceExtension;
+ p_port_ext = p_pdo[num_pdo]->DeviceExtension;
p_port_ext->pdo.dev_po_state.DeviceState = PowerDeviceD0;
p_port_ext->pdo.p_parent_ext = bus_globals.p_bus_ext;
p_port_ext->pdo.b_present = TRUE;
@@ -615,22 +620,33 @@
p_port_ext->pdo.b_hibernating = FALSE;
p_port_ext->pdo.p_po_work_item = NULL;
BUS_TRACE( BUS_DBG_PNP, ("Created device for %s: PDO %p,ext
%p, present %d, missing %d .\n",
- p_port_ext->pdo.cl_ext.vfptr_pnp_po->identity,
p_pdo, p_port_ext, p_port_ext->pdo.b_present,
+ p_port_ext->pdo.cl_ext.vfptr_pnp_po->identity,
p_pdo[num_pdo], p_port_ext, p_port_ext->pdo.b_present,
p_port_ext->pdo.b_reported_missing ) );
/* Cache the CA GUID. */
p_port_ext->pdo.ca_guid = p_pnp_rec->p_ca_attr->ca_guid;
/* Take a reference on the parent HCA. */
- p_port_ext->pdo.h_ca = acquire_ca(
p_pnp_rec->p_ca_attr->ca_guid );
+ if(num_pdo > 0)
+ {
+ p_port_ext->pdo.h_ca =
((bus_port_ext_t*)p_pdo[0]->DeviceExtension)->pdo.h_ca;
+ }
+ else
+ p_port_ext->pdo.h_ca = acquire_ca(
p_pnp_rec->p_ca_attr->ca_guid );
if( !p_port_ext->pdo.h_ca )
{
BUS_TRACE( BUS_DBG_PNP, ("Deleted device: PDO
%p\n", p_pdo ));
- IoDeleteDevice( p_pdo );
+ IoDeleteDevice( p_pdo[num_pdo] );
BUS_TRACE_EXIT( BUS_DBG_ERROR, ("acquire_ca
failed to find CA.\n") );
return IB_INVALID_GUID;
}
- p_port_ext->port_guid = p_pnp_rec->p_port_attr->port_guid;
+ p_port_ext->port_guid.port_guid =
p_pnp_rec->p_port_attr->port_guid;
+ p_port_ext->port_guid.pkey = IB_DEFAULT_PKEY;
+
+ if(num_pdo > 0)
+ {
+ p_port_ext->port_guid.pkey =
g_ib_pkey_array.pKey[num_pdo -1];
+ }
p_port_ext->n_port = p_pnp_rec->p_port_attr->port_num;
/* Store the device extension in the port vector for future
queries. */
@@ -643,8 +659,11 @@
* Set the context of the PNP event. The context is passed
in for future
* events on the same port.
*/
- p_pnp_rec->pnp_rec.context = p_port_ext;
+ if(num_pdo == 0)
+ p_pnp_rec->pnp_rec.context = p_port_ext;
+ }
+
/* Tell the PnP Manager to rescan for the HCA's bus
relations. */
IoInvalidateDeviceRelations(
p_port_ext->pdo.h_ca->obj.p_ci_ca->verbs.p_hca_dev, BusRelations );
Index: core/bus/kernel/ib_bus.inf
===================================================================
--- core/bus/kernel/ib_bus.inf (revision 1026)
+++ core/bus/kernel/ib_bus.inf (working copy)
@@ -183,6 +183,7 @@
HKR,"Parameters","IocQueryTimeout",%REG_DWORD_NO_CLOBBER%,250
HKR,"Parameters","IocQueryRetries",%REG_DWORD_NO_CLOBBER%,4
HKR,"Parameters","IocPollInterval",%REG_DWORD_NO_CLOBBER%,30000
+HKR,"Parameters","IbPkey",%REG_SZ%,"0xBABA"
[Iou.ParamsReg]
HKR,"Parameters","DebugLevel",%REG_DWORD%,2
Index: ulp/ipoib/kernel/ipoib_adapter.c
===================================================================
--- ulp/ipoib/kernel/ipoib_adapter.c (revision 1026)
+++ ulp/ipoib/kernel/ipoib_adapter.c (working copy)
@@ -171,7 +171,7 @@
IPOIB_PRINT( TRACE_LEVEL_INFORMATION, IPOIB_DBG_INIT,
("Port %016I64x (CA %016I64x port
%d) initializing\n",
- p_adapter->guids.port_guid,
p_adapter->guids.ca_guid,
+ p_adapter->guids.port_guid.port_guid,
p_adapter->guids.ca_guid,
p_adapter->guids.port_num) );
cl_status = cl_obj_init( &p_adapter->obj, CL_DESTROY_SYNC,
@@ -343,7 +343,7 @@
/* Validate the port GUID and generate the MAC address. */
status =
- ipoib_mac_from_guid( p_adapter->guids.port_guid,
&p_adapter->mac );
+ ipoib_mac_from_guid(
p_adapter->guids.port_guid.port_guid, &p_adapter->mac );
if( status != IB_SUCCESS )
{
IPOIB_PRINT_EXIT( TRACE_LEVEL_ERROR,
IPOIB_DBG_ERROR,
@@ -530,7 +530,7 @@
case IB_PNP_PORT_ADD:
CL_ASSERT( !p_pnp_rec->context );
/* Only process our port GUID. */
- if( p_pnp_rec->guid !=
p_adapter->guids.port_guid )
+ if( p_pnp_rec->guid !=
p_adapter->guids.port_guid.port_guid )
{
status = IB_NOT_DONE;
break;
Index: ulp/ipoib/kernel/ipoib_driver.c
===================================================================
--- ulp/ipoib/kernel/ipoib_driver.c (revision 1026)
+++ ulp/ipoib/kernel/ipoib_driver.c (working copy)
@@ -2336,10 +2336,10 @@
cl_memset( &ib_service, 0, sizeof(ib_service) );
/* BUGBUG Only register local subnet GID prefix for now */
- ib_gid_set_default( &port_gid, p_adapter->guids.port_guid );
+ ib_gid_set_default( &port_gid,
p_adapter->guids.port_guid.port_guid );
ib_service.svc_rec.service_gid = port_gid;
- ib_service.svc_rec.service_pkey =
IB_DEFAULT_PKEY;
+ ib_service.svc_rec.service_pkey =
p_adapter->guids.port_guid.pkey;
ib_service.svc_rec.service_lease =
IB_INFINITE_SERVICE_LEASE;
/* Must cast here because the service name is an array of
unsigned chars but
@@ -2347,7 +2347,7 @@
strcpy( (char *)ib_service.svc_rec.service_name, ATS_NAME );
/* IP Address in question will be put in below */
- ib_service.port_guid =
p_adapter->guids.port_guid;
+ ib_service.port_guid =
p_adapter->guids.port_guid.port_guid;
ib_service.timeout_ms =
p_adapter->params.sa_timeout;
ib_service.retry_cnt =
p_adapter->params.sa_retry_cnt;
Index: ulp/ipoib/kernel/ipoib_endpoint.c
===================================================================
--- ulp/ipoib/kernel/ipoib_endpoint.c (revision 1026)
+++ ulp/ipoib/kernel/ipoib_endpoint.c (working copy)
@@ -312,13 +312,13 @@
cl_memclr( &path, sizeof(ib_path_rec_t) );
path.dgid = p_endpt->dgid;
- ib_gid_set_default( &path.sgid,
p_port->p_adapter->guids.port_guid );
+ ib_gid_set_default( &path.sgid,
p_port->p_adapter->guids.port_guid.port_guid );
path.num_path = 0x1;
cl_memclr( &query, sizeof(ib_query_req_t) );
query.query_type = IB_QUERY_USER_DEFINED;
query.p_query_input = &info;
- query.port_guid = p_port->p_adapter->guids.port_guid;
+ query.port_guid =
p_port->p_adapter->guids.port_guid.port_guid;
query.timeout_ms = p_port->p_adapter->params.sa_timeout;
query.retry_cnt = p_port->p_adapter->params.sa_retry_cnt;
Index: ulp/ipoib/kernel/ipoib_ibat.c
===================================================================
--- ulp/ipoib/kernel/ipoib_ibat.c (revision 1026)
+++ ulp/ipoib/kernel/ipoib_ibat.c (working copy)
@@ -140,7 +140,7 @@
{
pAdapter = CONTAINING_RECORD( pItem,
ipoib_adapter_t, entry );
pOut->Ports[pOut->NumPorts].CaGuid =
pAdapter->guids.ca_guid;
- pOut->Ports[pOut->NumPorts].PortGuid =
pAdapter->guids.port_guid;
+ pOut->Ports[pOut->NumPorts].PortGuid =
pAdapter->guids.port_guid.port_guid;
pOut->Ports[pOut->NumPorts].PortNum =
pAdapter->guids.port_num;
pOut->NumPorts++;
@@ -210,7 +210,7 @@
pItem = cl_qlist_next( pItem ) )
{
pAdapter = CONTAINING_RECORD( pItem,
ipoib_adapter_t, entry );
- if( PortGuid && pAdapter->guids.port_guid !=
PortGuid )
+ if( PortGuid &&
pAdapter->guids.port_guid.port_guid != PortGuid )
continue;
cl_obj_lock( &pAdapter->obj );
@@ -300,7 +300,7 @@
pItem = cl_qlist_next( pItem ) )
{
pAdapter = CONTAINING_RECORD( pItem,
ipoib_adapter_t, entry );
- if( pIn->PortGuid != pAdapter->guids.port_guid )
+ if( pIn->PortGuid !=
pAdapter->guids.port_guid.port_guid )
continue;
/* Found the port - lookup the MAC. */
Index: ulp/ipoib/kernel/ipoib_port.c
===================================================================
--- ulp/ipoib/kernel/ipoib_port.c (revision 1026)
+++ ulp/ipoib/kernel/ipoib_port.c (working copy)
@@ -793,6 +793,8 @@
uint64_t vaddr;
net32_t rkey;
ib_qp_attr_t qp_attr;
+ ib_ca_attr_t *ca_attr;
+ uint32_t ca_size;
IPOIB_ENTER( IPOIB_DBG_INIT );
@@ -810,6 +812,44 @@
return status;
}
+ /* Query the CA for Pkey table */
+ status =
p_port->p_adapter->p_ifc->query_ca(p_port->ib_mgr.h_ca, NULL, &ca_size);
+ if(status != IB_INSUFFICIENT_MEMORY)
+ {
+ IPOIB_PRINT_EXIT( TRACE_LEVEL_ERROR,
IPOIB_DBG_ERROR,
+ ("ib_query_ca failed\n"));
+ return status;
+ }
+
+ ca_attr = (ib_ca_attr_t*)cl_zalloc(ca_size);
+ if (!ca_attr)
+ {
+ IPOIB_PRINT_EXIT( TRACE_LEVEL_ERROR,
IPOIB_DBG_ERROR,
+ ("cl_zalloc can't allocate
%d\n",ca_size));
+ return IB_INSUFFICIENT_RESOURCES;
+ }
+
+ status =
p_port->p_adapter->p_ifc->query_ca(p_port->ib_mgr.h_ca,
ca_attr,&ca_size);
+ if( status != IB_SUCCESS )
+ {
+ IPOIB_PRINT_EXIT( TRACE_LEVEL_ERROR,
IPOIB_DBG_ERROR,
+ ("ib_query_ca returned %s\n",
+
p_port->p_adapter->p_ifc->get_err_str( status )) );
+ return status;
+ }
+ if( ca_attr->p_port_attr->link_state == IB_LINK_ACTIVE)
+ {
+ uint16_t index;
+ CL_ASSERT(ca_attr->p_port_attr->p_pkey_table[0]
== IB_DEFAULT_PKEY);
+ for(index = 0; index <
ca_attr->p_port_attr->num_pkeys; index++)
+ {
+
if(p_port->p_adapter->guids.port_guid.pkey ==
ca_attr->p_port_attr->p_pkey_table[index])
+ break;
+ }
+ p_port->pkey_index = index;
+ }
+ cl_free(ca_attr);
+
/* Allocate the PD. */
status = p_port->p_adapter->p_ifc->alloc_pd(
p_port->ib_mgr.h_ca, IB_PDT_UD, p_port,
&p_port->ib_mgr.h_pd );
@@ -3309,7 +3349,7 @@
CL_ASSERT( p_cid[1] == 21 );
p_cid[23]= DHCP_OPT_END;
- ib_gid_set_default( &gid,
p_port->p_adapter->guids.port_guid );
+ ib_gid_set_default( &gid,
p_port->p_adapter->guids.port_guid.port_guid );
cl_memcpy( &p_cid[7], &gid, sizeof(ib_gid_t) );
cl_memcpy( &p_cid[3], &p_port->ib_mgr.qpn,
sizeof(p_port->ib_mgr.qpn) );
p_ib_dhcp->htype = DHCP_HW_TYPE_IB;
@@ -3409,7 +3449,7 @@
p_ib_arp->op = p_arp->op;
p_ib_arp->src_hw.flags_qpn = p_port->ib_mgr.qpn;
ib_gid_set_default( &p_ib_arp->src_hw.gid,
- p_port->p_adapter->guids.port_guid );
+ p_port->p_adapter->guids.port_guid.port_guid );
p_ib_arp->src_ip = p_arp->src_ip;
if( cl_memcmp( &p_arp->dst_hw, &null_hw, sizeof(mac_addr_t)
) )
{
@@ -3584,7 +3624,7 @@
p_desc->wr.dgrm.ud.remote_qp = p_desc->p_endpt->qpn;
p_desc->wr.dgrm.ud.remote_qkey =
p_port->ib_mgr.bcast_rec.qkey;
p_desc->wr.dgrm.ud.h_av = p_desc->p_endpt->h_av;
- p_desc->wr.dgrm.ud.pkey_index = 0;
+ p_desc->wr.dgrm.ud.pkey_index = p_port->pkey_index;
p_desc->wr.dgrm.ud.rsvd = NULL;
/* Store context in our reserved area of the packet. */
@@ -4683,7 +4723,7 @@
cl_memclr( &query, sizeof(ib_query_req_t) );
query.query_type = IB_QUERY_USER_DEFINED;
query.p_query_input = &info;
- query.port_guid = p_port->p_adapter->guids.port_guid;
+ query.port_guid =
p_port->p_adapter->guids.port_guid.port_guid;
query.timeout_ms = p_port->p_adapter->params.sa_timeout;
query.retry_cnt = p_port->p_adapter->params.sa_retry_cnt;
query.query_context = p_port;
@@ -4721,7 +4761,7 @@
IPOIB_ENTER( IPOIB_DBG_INIT );
- ib_gid_set_default( &gid, p_port->p_adapter->guids.port_guid
);
+ ib_gid_set_default( &gid,
p_port->p_adapter->guids.port_guid.port_guid );
p_endpt = ipoib_endpt_create(
&gid, p_port_info->base_lid, p_port->ib_mgr.qpn
);
if( !p_endpt )
@@ -4894,10 +4934,13 @@
cl_memclr( &member_rec, sizeof(ib_member_rec_t) );
member_rec.mgid = bcast_mgid_template;
+ member_rec.mgid.raw[4] =
(uint8_t)(p_port->p_adapter->guids.port_guid.pkey >> 8);
+ member_rec.mgid.raw[5] = (uint8_t)
p_port->p_adapter->guids.port_guid.pkey;
+ member_rec.pkey = p_port->p_adapter->guids.port_guid.pkey;
cl_memclr( &query, sizeof(ib_query_req_t) );
query.query_type = IB_QUERY_USER_DEFINED;
query.p_query_input = &info;
- query.port_guid = p_port->p_adapter->guids.port_guid;
+ query.port_guid =
p_port->p_adapter->guids.port_guid.port_guid;
query.timeout_ms = p_port->p_adapter->params.sa_timeout;
query.retry_cnt = p_port->p_adapter->params.sa_retry_cnt;
query.query_context = p_port;
@@ -5031,13 +5074,13 @@
/* We specify our port GID for the join operation. */
mcast_req.member_rec.port_gid.unicast.prefix =
IB_DEFAULT_SUBNET_PREFIX;
mcast_req.member_rec.port_gid.unicast.interface_id =
- p_port->p_adapter->guids.port_guid;
+ p_port->p_adapter->guids.port_guid.port_guid;
mcast_req.mcast_context = p_port;
mcast_req.pfn_mcast_cb = __bcast_cb;
mcast_req.timeout_ms = p_port->p_adapter->params.sa_timeout;
mcast_req.retry_cnt =
p_port->p_adapter->params.sa_retry_cnt;
- mcast_req.port_guid = p_port->p_adapter->guids.port_guid;
+ mcast_req.port_guid =
p_port->p_adapter->guids.port_guid.port_guid;
mcast_req.pkey_index = 0;
if( ib_member_get_state( mcast_req.member_rec.scope_state )
!=
@@ -5086,7 +5129,9 @@
*/
mcast_req.member_rec.mgid = bcast_mgid_template;
ib_gid_set_default( &mcast_req.member_rec.port_gid,
- p_port->p_adapter->guids.port_guid );
+ p_port->p_adapter->guids.port_guid.port_guid );
+ mcast_req.member_rec.mgid.raw[4] = (uint8_t)
(p_port->p_adapter->guids.port_guid.pkey >> 8);
+ mcast_req.member_rec.mgid.raw[5] = (uint8_t)
p_port->p_adapter->guids.port_guid.pkey;
/*
* IPOIB spec requires that the QKEY have the MSb set so
that the QKEY
* from the QP is used rather than the QKEY in the send WR.
@@ -5096,7 +5141,7 @@
mcast_req.member_rec.mtu =
(IB_PATH_SELECTOR_EXACTLY << 6) |
IB_MTU_LEN_2048;
- mcast_req.member_rec.pkey = IB_DEFAULT_PKEY;
+ mcast_req.member_rec.pkey =
p_port->p_adapter->guids.port_guid.pkey;
mcast_req.member_rec.sl_flow_hop =
ib_member_set_sl_flow_hop( 0, 0, 0 );
mcast_req.member_rec.scope_state =
@@ -5106,8 +5151,8 @@
mcast_req.pfn_mcast_cb = __bcast_cb;
mcast_req.timeout_ms = p_port->p_adapter->params.sa_timeout;
mcast_req.retry_cnt =
p_port->p_adapter->params.sa_retry_cnt;
- mcast_req.port_guid = p_port->p_adapter->guids.port_guid;
- mcast_req.pkey_index = 0;
+ mcast_req.port_guid =
p_port->p_adapter->guids.port_guid.port_guid;
+ mcast_req.pkey_index = p_port->pkey_index;
/* reference the object for the multicast join request. */
ipoib_port_ref( p_port, ref_join_bcast );
@@ -5362,9 +5407,9 @@
}
/* Move the QP to RTS. */
- dgrm_info.port_guid = p_port->p_adapter->guids.port_guid;
+ dgrm_info.port_guid =
p_port->p_adapter->guids.port_guid.port_guid;
dgrm_info.qkey = p_port->ib_mgr.bcast_rec.qkey;
- dgrm_info.pkey_index = 0;
+ dgrm_info.pkey_index = p_port->pkey_index;
status = p_port->p_adapter->p_ifc->init_dgrm_svc(
p_port->ib_mgr.h_qp, &dgrm_info );
if( status != IB_SUCCESS )
{
@@ -5464,9 +5509,9 @@
mcast_req.pfn_mcast_cb = __mcast_cb;
mcast_req.timeout_ms = p_port->p_adapter->params.sa_timeout;
mcast_req.retry_cnt =
p_port->p_adapter->params.sa_retry_cnt;
- mcast_req.port_guid = p_port->p_adapter->guids.port_guid;
- mcast_req.pkey_index = 0;
-
+ mcast_req.port_guid =
p_port->p_adapter->guids.port_guid.port_guid;
+ mcast_req.pkey_index = p_port->pkey_index;
+ mcast_req.member_rec.pkey =
p_port->p_adapter->guids.port_guid.pkey;
/*
* Create the endpoint and insert it in the port. Since we
don't wait for
* the mcast SA operations to complete before returning from
the multicast
Index: ulp/ipoib/kernel/ipoib_port.h
===================================================================
--- ulp/ipoib/kernel/ipoib_port.h (revision 1026)
+++ ulp/ipoib/kernel/ipoib_port.h (working copy)
@@ -506,6 +506,7 @@
atomic32_t
endpt_rdr;
atomic32_t
hdr_idx;
+ uint16_t pkey_index;
ipoib_hdr_t
hdr[1]; /* Must be last! */
} ipoib_port_t;
Slava Strebkov
SW Engineer
Voltaire
099718750
------------------------------
_______________________________________________
ofw mailing list
ofw at lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
End of ofw Digest, Vol 15, Issue 12
***********************************
More information about the ofw
mailing list